Client sends a HTTP POST request with JSON payload and content type "application/json".
Backend REST service that is proxied by NGINX (via Ingress Controller in k8s) receives the request with content type "application/xml" and the JSON is converted to a weird XML message. Below is my ingress object which has client auth turned on for mTLS:
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
nginx.ingress.kubernetes.io/auth-tls-secret: xyz/xyz-secret
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "2"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: xyz-ingress
namespace: xyz
spec:
ingressClassName: nginx
rules:
- host: host_name
http:
paths:
- backend:
service:
name: api
port:
number: 80
path: /xyz/api
pathType: Prefix
tls:
- hosts:
- host_name
secretName: xyz-secret```
This is the request payload in XML received at the backend rest server:
```<.Proxy209 xmlns=""><tenant>xxx</tenant><serviceType>xxx</serviceType></.Proxy209>
The client sends out this JSON:
```{ "tenant": "xxx", "serviceType": "xxx" }```
The JSON is converted to the XML above by NGINX.
Related
We would like to use annotation for redirecting requests to a different backend service based on url args (query)
Example:
https://example.com/foo?differentQueryString=0 -> service-a
https://example.com/foo/bar?queryString=0 - service-b
Notes: path does not matter, this can be either /foo/bar or /foo or /bar/foo
We followed up on this
Kubernetes NGINX Ingress controller - different route if query string exists
and this
Kubernetes ingress routes with url parameter
But we don't want to setup ConfigMap just for this and also we don't want to duplicate requests to the ingress but rewriting
This is what we tried
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
backend.service.name = service-b
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
We were expecting to get the response but we got 502 from the Ingress Nginx
We managed to find a nice solution without rewriting and ConfigMap
Works great and also include Nginx Ingress metrics so we can do HPA accordingly
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
set $proxy_upstream_name "default-service-b-80";
set $proxy_host $proxy_upstream_name;
set $service_name "service-b";
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
$proxy_upsteam_name convention is NAMESPACE-SERVICE_NAME-PORT
I've deployed Nginx Ingress Controller, manifest:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: namespace-servicename
name: servicename
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-cf-dns"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- roabc.domain.net
secretName: wildcard-roudh-domain-net
ingressClassName: public
rules:
- host: roabc.domain.net
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: servicename
port:
number: 80
kubectl describe svc servicename -n namespace-servicename
Name: servicename
Namespace: namespace-servicename
Labels: app=ro**
continent=eu
hub-id=eu-**-mad013
Annotations: <none>
Selector: app=ro**,continent=eu,hub-id=eu-**-mad013
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.152.183.44
IPs: 10.152.183.44
Port: grpc 80/TCP
TargetPort: 80/TCP
Endpoints: 10.1.252.108:80
Session Affinity: None
Events: <none>
But, unfortunately, grpcurl can't reach gRPC application, throws error rpc error: code = Internal desc = failed to query for service descriptor"*.services..SearchList":stream terminated by RST_STREAM with error code: PROTOCOL_ERROR
grpcurl -d "#" -H "authorization: Bt** dert**" roabc.domain.net:443
Error invoking method"***.services.**.SearchList": rpc error: code = Internal desc = failed to query for service descriptor"***.services.**.SearchList": **stream terminated by RST_STREAM with error code: PROTOCOL_ERROR**
gRPC app running in a Pod, constantly gives this error: OPENSSL_internal:WRONG_VERSION_NUMBER
**ssl_transport_security.cc:1455] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.**
If you prefer to forward encrypted traffic to your POD and terminate TLS at the gRPC server itself, add the ingress annotation
nginx.ingress.kubernetes.io/backend-protocol: "GRPCS"
Context: I have an application running on GKE. The application has 3 APIs. I have an Ingress load balancer that exposes an external IP for my front-end to call, and depending on the path will forward the request to appropriate APIs.
Problem: Calling the individual path returns a "404 Not Found - nginx" message. I call using a proxy (request format: https://my-proxy-hosted-on-heroku/http://xx.xxx.xxx.xx/api/auth/accounts/login/) (xx.xxx.xxx.xx is the Ingress external IP)
My Ingress yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
nginx.ingress.kubernetes.io/cors-expose-headers: "*, X-CustomResponseHeader"
# nginx.ingress.kubernetes.io/ssl-redirect: "false"
# nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
# kubernetes.io/ingress.class: "nginx"
# nginx.ingress.kubernetes.io/enable-cors: "true"
# nginx.ingress.kubernetes.io/cors-allow-origin: "*"
# nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
# nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
spec:
rules:
- host: acme.com
http:
paths:
- pathType: Prefix
path: /api/discussion
backend:
service:
name: fplms-discussionservice-clusterip
port:
number: 7218
- pathType: Prefix
path: /api/auth
backend:
service:
name: fplms-authservice-clusterip
port:
number: 7209
- pathType: Prefix
path: /api/management
backend:
service:
name: fplms-managementservice-clusterip
port:
number: 8083
I have this base url api.example.com
So, ingress-nginx will get the request for api.example.com and it should do follow things.
Forward api.example.com/customer to customer-srv
It doesn't work, it forwards whole mtach to customer-srv i.e. /customer/requested_url
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
I tried using rewrite annotation but that doesn't work either however this worked but this is not I want to achieve.
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For example,
api.example.com/customer should go to http://localhost:3000 not http://localhost:3000/customer
Here is the yaml I used:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For test purpouses I created an echo server:
kubectl run --image mendhak/http-https-echo customer
And then a service:
kubectl expose po customer --name customer-srv --port 3000 --target-port 80
I checked igress ip:
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-service <none> api.example.in 192.168.39.254 80 3m43s
And I did a curl to check it:
curl 192.168.39.254/customer/asd -H "Host: api.example.in"
{
"path": "/asd",
"headers": {
"host": "api.example.in",
...
}
Notice that the echo server echoed back a path that it received, and sice it received a path that got rewritten from /customer/asd to /asd it shows this exectly path (/asd).
So as you see this does work.
I have been trying to apply this virtual service yaml for my microservices:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nameko-notifyms
spec:
hosts:
- "*"
gateways:
- nameko-notifyms-gateway
http:
- match:
- headers:
cookie:
regex: "^(.*?;)?(user=joe)(;.*)?"
uri:
exact: /hello
route:
- destination:
host: nameko-notifyms
port:
number: 8000
Using the code block above, after curl-ing the uri, there is not traffic going into the pod.
If I comment out the information as shown in the code block below:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nameko-notifyms
spec:
hosts:
- "*"
gateways:
- nameko-notifyms-gateway
http:
- match:
# - headers:
# cookie:
# regex: "^(.*?;)?(user=joe)(;.*)?"
- uri:
exact: /hello
route:
- destination:
host: nameko-notifyms
port:
number: 8000
The traffic is directed to the pod, which can be shown in the image below:
Postman setting as below:
Hello the problem is old but still actual so here the solution :
The problem come from the regex the first block ; is not optional.
Here the corrected regex.
"^(.*;?)?(user=joe)(;.*)?"
Full detail here : https://regex101.com/r/CPv2kU/3