I've deployed Nginx Ingress Controller, manifest:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: namespace-servicename
name: servicename
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-cf-dns"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- roabc.domain.net
secretName: wildcard-roudh-domain-net
ingressClassName: public
rules:
- host: roabc.domain.net
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: servicename
port:
number: 80
kubectl describe svc servicename -n namespace-servicename
Name: servicename
Namespace: namespace-servicename
Labels: app=ro**
continent=eu
hub-id=eu-**-mad013
Annotations: <none>
Selector: app=ro**,continent=eu,hub-id=eu-**-mad013
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.152.183.44
IPs: 10.152.183.44
Port: grpc 80/TCP
TargetPort: 80/TCP
Endpoints: 10.1.252.108:80
Session Affinity: None
Events: <none>
But, unfortunately, grpcurl can't reach gRPC application, throws error rpc error: code = Internal desc = failed to query for service descriptor"*.services..SearchList":stream terminated by RST_STREAM with error code: PROTOCOL_ERROR
grpcurl -d "#" -H "authorization: Bt** dert**" roabc.domain.net:443
Error invoking method"***.services.**.SearchList": rpc error: code = Internal desc = failed to query for service descriptor"***.services.**.SearchList": **stream terminated by RST_STREAM with error code: PROTOCOL_ERROR**
gRPC app running in a Pod, constantly gives this error: OPENSSL_internal:WRONG_VERSION_NUMBER
**ssl_transport_security.cc:1455] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.**
If you prefer to forward encrypted traffic to your POD and terminate TLS at the gRPC server itself, add the ingress annotation
nginx.ingress.kubernetes.io/backend-protocol: "GRPCS"
Related
We would like to use annotation for redirecting requests to a different backend service based on url args (query)
Example:
https://example.com/foo?differentQueryString=0 -> service-a
https://example.com/foo/bar?queryString=0 - service-b
Notes: path does not matter, this can be either /foo/bar or /foo or /bar/foo
We followed up on this
Kubernetes NGINX Ingress controller - different route if query string exists
and this
Kubernetes ingress routes with url parameter
But we don't want to setup ConfigMap just for this and also we don't want to duplicate requests to the ingress but rewriting
This is what we tried
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
backend.service.name = service-b
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
We were expecting to get the response but we got 502 from the Ingress Nginx
We managed to find a nice solution without rewriting and ConfigMap
Works great and also include Nginx Ingress metrics so we can do HPA accordingly
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
set $proxy_upstream_name "default-service-b-80";
set $proxy_host $proxy_upstream_name;
set $service_name "service-b";
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
$proxy_upsteam_name convention is NAMESPACE-SERVICE_NAME-PORT
Context: I have an application running on GKE. The application has 3 APIs. I have an Ingress load balancer that exposes an external IP for my front-end to call, and depending on the path will forward the request to appropriate APIs.
Problem: Calling the individual path returns a "404 Not Found - nginx" message. I call using a proxy (request format: https://my-proxy-hosted-on-heroku/http://xx.xxx.xxx.xx/api/auth/accounts/login/) (xx.xxx.xxx.xx is the Ingress external IP)
My Ingress yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
nginx.ingress.kubernetes.io/cors-expose-headers: "*, X-CustomResponseHeader"
# nginx.ingress.kubernetes.io/ssl-redirect: "false"
# nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
# kubernetes.io/ingress.class: "nginx"
# nginx.ingress.kubernetes.io/enable-cors: "true"
# nginx.ingress.kubernetes.io/cors-allow-origin: "*"
# nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
# nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
spec:
rules:
- host: acme.com
http:
paths:
- pathType: Prefix
path: /api/discussion
backend:
service:
name: fplms-discussionservice-clusterip
port:
number: 7218
- pathType: Prefix
path: /api/auth
backend:
service:
name: fplms-authservice-clusterip
port:
number: 7209
- pathType: Prefix
path: /api/management
backend:
service:
name: fplms-managementservice-clusterip
port:
number: 8083
I have this base url api.example.com
So, ingress-nginx will get the request for api.example.com and it should do follow things.
Forward api.example.com/customer to customer-srv
It doesn't work, it forwards whole mtach to customer-srv i.e. /customer/requested_url
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
I tried using rewrite annotation but that doesn't work either however this worked but this is not I want to achieve.
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For example,
api.example.com/customer should go to http://localhost:3000 not http://localhost:3000/customer
Here is the yaml I used:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For test purpouses I created an echo server:
kubectl run --image mendhak/http-https-echo customer
And then a service:
kubectl expose po customer --name customer-srv --port 3000 --target-port 80
I checked igress ip:
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-service <none> api.example.in 192.168.39.254 80 3m43s
And I did a curl to check it:
curl 192.168.39.254/customer/asd -H "Host: api.example.in"
{
"path": "/asd",
"headers": {
"host": "api.example.in",
...
}
Notice that the echo server echoed back a path that it received, and sice it received a path that got rewritten from /customer/asd to /asd it shows this exectly path (/asd).
So as you see this does work.
For nginx ingress, there is a way to define default-ssl-certificate with --default-ssl-certificate flag.
Ref: https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-ssl-certificate
How can I do the same for istio?
I have assigned tls.credentialName in istio gateway. But, it's not the same as nginx-ingress default-ssl-certificate.
istio_gateway.yaml
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: SERVICE_GATEWAY
spec:
selector:
istio: ingressgateway # Use Istio default gateway implementation
servers:
- port:
name: SERVICE_NAME-http-80
number: 80
protocol: HTTP
hosts:
- "SERVICE_DNS"
- port:
name: SERVICE_NAME-https-443
number: 443
protocol: HTTPS
tls:
credentialName: SERVICE_CRT
mode: SIMPLE
minProtocolVersion: TLSV1_2
hosts:
- "SERVICE_DNS"
VirtualService:
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: SERVICE_NAME
spec:
hosts:
- SERVICE_DNS
gateways:
- SERVICE_GATEWAY
http:
- match:
- uri:
prefix: /
route:
- destination:
port:
number: SERVICE_PORT
host: "SERVICE_NAME.default.svc.cluster.local"
This setup is working for nginx-ingress: https://ssbkang.com/2020/08/17/end-to-end-tls-for-azure-front-door-and-azure-kubernetes-service/
I want to do the same thing with istio.
I have a pod that responds to requests to /api/
I want to do a rewrite where requests to /auth/api/ go to /api/.
Using an Ingress (nginx), I thought that with the ingress.kubernetes.io/rewrite-target: annotation I could do it something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api
backend:
serviceName: myapi
servicePort: myapi-port
What's happening however is that /auth/ is being passed to the service/pod and a 404 is rightfully being thrown. I must be misunderstanding the rewrite annotation.
Is there a way to do this via k8s & ingresses?
I don't know if this is still an issue, but since version 0.22 it seems you need to use capture groups to pass values to the rewrite-target value
From the nginx example available here
Starting in Version 0.22.0, ingress definitions using the annotation nginx.ingress.kubernetes.io/rewrite-target are not backwards compatible with previous versions. In Version 0.22.0 and beyond, any substrings within the request URI that need to be passed to the rewritten path must explicitly be defined in a capture group.
For your specific needs, something like this should do the trick
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api/$2
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api(/|$)(.*)
backend:
serviceName: myapi
servicePort: myapi-port
I have created the following example that works and which I will explain. To run this minimal example, run these commands:
$ minikube start
$ minikube addons enable ingress # might take a while for ingress pod to bootstrap
$ kubectl apply -f kubernetes.yaml
$ curl https://$(minikube ip)/auth/api/ --insecure
success - path: /api/
$ curl https://$(minikube ip)/auth/api --insecure
failure - path: /auth/api
$ curl https://$(minikube ip)/auth/api/blah/whatever --insecure
success - path: /api/blah/whatever
As you'll notice, the ingress rewrite annotation appears to be very particular about trailing slashes. If a trailing slash is not present, the request will not be rewritten. However, if a trailing slash is provided, the request uri will be rewritten and your proxy will function as expected.
After inspecting the generated nginx.conf file from inside the ingress controller, the line of code responsible for this behavior is:
rewrite /auth/api/(.*) api/$1 break;
This line tells us that only requests matching the first argument will be rewritten with the path specified by the second argument.
I believe this is bug worthy.
kubernetes.yaml
---
apiVersion: v1
kind: Service
metadata:
name: ingress-rewite-example
spec:
selector:
app: ingress-rewite-example
ports:
- name: nginx
port: 80
protocol: TCP
targetPort: 80
type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ingress-rewite-example
spec:
template:
metadata:
labels:
app: ingress-rewite-example
spec:
containers:
- name: ingress-rewite-example
image: fbgrecojr/office-hours:so-47837087
imagePullPolicy: Always
ports:
- containerPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rewite-example
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /auth/api
backend:
serviceName: ingress-rewite-example
servicePort: 80
main.go
package main
import (
"fmt"
"strings"
"net/http"
)
func httpHandler(w http.ResponseWriter, r *http.Request) {
var response string
if strings.HasPrefix(r.URL.Path, "/api") {
response = "success"
} else {
response = "failure"
}
fmt.Fprintf(w, response + " - path: " + r.URL.Path + "\n")
}
func main() {
http.HandleFunc("/", httpHandler)
panic(http.ListenAndServe(":80", nil))
}