I have GRPC service written in go and i need to deploy the service on top of AWS-EKS, we are using nginx-ingress and cloudflare to point to our cluster gateway (nginx).
but when i tried to deploy the service and test it using this command grpcurl grpc.fd-admin.com:443 list
i always get the following error:
Failed to list services: rpc error: code = Unknown desc = Moved Permanently: HTTP status code 301; transport: missing content-type field
And this is what i did for kubernetes resources:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: grpc
labels:
k8s-app: grpc
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: grpc
spec:
containers:
- name: grpc
image: quay.io/kubernetes-ingress-controller/grpc-fortune-teller:0.1
ports:
- containerPort: 50051
name: grpc
---
apiVersion: v1
kind: Service
metadata:
name: grpc
namespace: grpc
spec:
selector:
k8s-app: grpc
ports:
- port: 50051
targetPort: 50051
protocol: TCP
name: grpc
type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: grpc
namespace: grpc
spec:
rules:
- host: grpc.fd-admin.com
http:
paths:
- backend:
serviceName: grpc
servicePort: grpc
tls:
- secretName: grpc
hosts:
- grpc.fd-admin.com
So can any one explain why i got this error or what is the reasons may cause this kind of error ?
Try below the annotations of nginx ingress,
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/grpc-backend-for-port: "grpc"
name: grpc
Related
Using nginx-ingress controller and metallb for my loadbalancer in my k3s raspberry pi cluster. Trying to access my blog site but I get white page with 400: Bad Request.
I'm using Cloudflare to managed my domain and SSL/TLS mode is on "Full". Created an A name "Blog" and pointed the content to my public external IP. I opened the Loadbalancer IP address on my router exposing 80 and 433. What am I missing. I've been pulling my hair with his issues for days now. Here's the example of my k3s entire deployment
apiVersion: v1
kind: Service
metadata:
namespace: nginx
name: nginx-web
labels:
app: nginx-web
spec:
ports:
# the port that this service should serve on
- port: 8000
targetPort: 80
protocol: TCP
selector:
app: nginx-web
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
replicas: 1
selector:
matchLabels:
app: nginx-web
template:
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
containers:
- name: nginx-web
image: nginx:latest
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
labels:
app: nginx-web
namespace: nginx
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- blog.example.com
secretName: blog-example-com-tls
rules:
- host: blog.example.com
http:
paths:
- backend:
service:
name: nginx-web
port:
number: 80
path: /
pathType: Prefix
I am working on a POC for Istio + gRPC, the Istio version is 1.6, but I could not see any gRPC traffic to my pods.
I suspect my Istio Gateway or VirtualService miss something, but I could not figure out what's wrong here? Could anybody help review my yaml file and correct me what's missing or wrong in my yaml?
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: syslogserver
name: syslogserver
namespace: mynamespace
spec:
selector:
matchLabels:
app: syslogserver
replicas: 1
template:
metadata:
labels:
app: syslogserver
spec:
containers:
- name: syslogserver
image: docker.io/grpc-syslog:latest
imagePullPolicy: Always
ports:
- containerPort: 5555
imagePullSecrets:
- name: pull-image-credential
---
apiVersion: v1
kind: Service
metadata:
name: syslogserver
namespace: mynamespace
labels:
app: syslogserver
spec:
selector:
app: syslogserver
ports:
- name: grpc
port: 6666
protocol: TCP
targetPort: 5555
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: xyz-ingress-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 7777
name: http2
protocol: HTTP2
hosts:
- "*"
---
apiVersion: v1
kind: Service
metadata:
name: xyz-istio-ingressgateway
namespace: istio-system
labels:
app: xyz-istio-ingressgateway
spec:
selector:
app: istio-ingressgateway
istio: ingressgateway
type: NodePort
ports:
- protocol: TCP
nodePort: 32555
port: 7777
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: xyz-ingress-gateway-virtualservice
namespace: istio-system
spec:
hosts:
- "*"
gateways:
- xyz-ingress-gateway
#tls:
http:
- match:
- port: 7777
route:
- destination:
host: syslogserver.mynamespace.svc.cluster.local
port:
number: 6666
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: xyz-destinationrule
namespace: istio-system
spec:
host: syslogserver.mynamespace.svc.cluster.local
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
Please give your guidance, thanks.
From what I see the service name: xyz-istio-ingressgateway should be deleted, as that's not how you communicate when using Istio.
Instead you should use istio ingress gateway, combined with a gateway, virtual service and destination rule.
If you've choosen port number 7777 on your gateway, you have to open this port on istio ingress gateway, there are few ways to do that in this stackoverflow question. There are the default istio ingress gateway values.
After you configure the port you can use kubectl get svc istio-ingressgateway -n istio-system to get the istio ingress gateway external IP.
If the external IP value is set, your environment has an external load balancer that you can use for the ingress gateway. If the EXTERNAL-IP value is pending, then your environment does not provide an external load balancer for the ingress gateway. In this case, you can access the gateway using the service’s node port.
The rest of your configuration looks fine for me. Just a reminder about injecting a sidecar proxy to your pods.
I have an issue I can't figure out. I have setup Nginx Ingress Controller on my managed k8s cluster. I'm trying to reach an SSL enabled pod behind and it does not work. I have 404 not found from Nginx and the certificate which is presented is the Nginx one. I have deployed the controller using their github repo and the default files following their doc.
I have setup a clear http pod for purpose tests and it works. It seems to be related to ssl.
I have tried many things to no avail. How can I reach an SSL pod behind nginx ?
Here's the Deployment + service (for the https one) resource I have setup :
apiVersion: apps/v1
kind: Deployment
metadata:
name: moulip-https
spec:
selector:
matchLabels:
app: moulip-https
replicas: 2
template:
metadata:
labels:
app: moulip-https
spec:
containers:
- name: "wabam"
image: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
ports:
- containerPort: 443
imagePullSecrets:
- name: regcrd
---
apiVersion: v1
kind: Service
metadata:
name: https-svc
labels:
app: moulip-https
spec:
ports:
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: moulip-https
and my Ingress :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: default
spec:
rules:
- host: https.moulip.lan
http:
paths:
- backend:
serviceName: https-svc
servicePort: 443
- host: test.moulip.lan
http:
paths:
- backend:
serviceName: hostname-svc
servicePort: 80
Many thanks for any guidance you could provide me with.
You are missing tls configuration in the ingress. follow sample below
apiVersion: v1
kind: Secret
metadata:
name: testsecret-tls
namespace: default
data:
tls.crt: base64 encoded cert
tls.key: base64 encoded key
type: kubernetes.io/tls
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: tls-example-ingress
spec:
tls:
- hosts:
- sslexample.foo.com
secretName: testsecret-tls
rules:
- host: sslexample.foo.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80
For the past two weeks, I've been working on deploying a gRPC service (also with a gRPC gateway). I got stuck a lot at many points and it still doesn't work... Why? I am not sure to know.
I first used Ingress from Google but it wasn't working with gRPC, so I moved to Ingress NGINX, but I still have issues...
I installed NGINX from https://kubernetes.github.io/ingress-nginx/deploy/#gce-gke
I added some annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/grpc-backend: "true"
service.yaml (containing my Service, Deployment, Ingress (gRPC) and Ingress (REST))
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: gkegrpcservice
namespace: default
spec:
replicas: 1
selector:
matchLabels:
run: gkegrpcservice
template:
metadata:
labels:
run: gkegrpcservice
spec:
containers:
- name: gkegrpcservice
image: gcr.io/<PROJECT_ID>/gkegrpcservice:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
- containerPort: 8081
---
apiVersion: v1
kind: Service
metadata:
annotations:
cloud.google.com/app-protocols: '{"grpc":"HTTP2"}'
name: gkegrpcservice
namespace: default
spec:
type: NodePort
selector:
run: gkegrpcservice
ports:
# Port that accepts gRPC and JSON/HTTP2 requests over HTTP.
- port: 8080
targetPort: 8080
protocol: TCP
name: grpc
# Port that accepts REST requests over HTTP.
- port: 8081
targetPort: 8081
protocol: TCP
name: rest
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gkegrpcservice-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/grpc-backend: "true"
namespace: default
spec:
rules:
- http:
paths:
- path: /GKEgRPCService.GKEgRPCService/*
backend:
serviceName: gkegrpcservice
servicePort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gkegrpcservice-gateway-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /rest/*
backend:
serviceName: gkegrpcservice
servicePort: 8081
I also created a github repository, so you can access the whole code/configuration but also the Dockerfile and the service.yaml: https://github.com/Emixam23/GKE-gRPC-Service-Ingress
based on the above, I then describe my Ingress:
MacBook-Pro-de-Emixam23:src emixam23$ kubectl describe ingress
Name: gkegrpcservice-gateway-ingress
Namespace: default
Address: 35.228.118.83
Default backend: default-http-backend:80 (10.4.2.7:8080)
Rules:
Host Path Backends
---- ---- --------
*
/rest/* gkegrpcservice:8081 (10.4.1.25:8081)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"gkegrpcservice-gateway-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"gkegrpcservice","servicePort":8081},"path":"/rest/*"}]}}]}}
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: false
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 26m nginx-ingress-controller Ingress default/gkegrpcservice-gateway-ingress
Normal UPDATE 115s (x3 over 26m) nginx-ingress-controller Ingress default/gkegrpcservice-gateway-ingress
Name: gkegrpcservice-ingress
Namespace: default
Address: 35.228.118.83
Default backend: default-http-backend:80 (10.4.2.7:8080)
Rules:
Host Path Backends
---- ---- --------
*
/GKEgRPCService.GKEgRPCService/* gkegrpcservice:8080 (10.4.1.25:8080)
Annotations:
nginx.ingress.kubernetes.io/backend-protocol: GRPC
nginx.ingress.kubernetes.io/grpc-backend: true
nginx.ingress.kubernetes.io/ssl-redirect: false
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/backend-protocol":"GRPC","nginx.ingress.kubernetes.io/grpc-backend":"true","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"gkegrpcservice-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"gkegrpcservice","servicePort":8080},"path":"/GKEgRPCService.GKEgRPCService/*"}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 6m43s nginx-ingress-controller Ingress default/gkegrpcservice-ingress
Normal UPDATE 6m6s nginx-ingress-controller Ingress default/gkegrpcservice-ingress
From here, 35.228.118.83/rest/health_check should return
200 {}
But only a 404 get returned...
However, this (to me) doesn't make sense, even by taking out /rest/, I should get a response, but OK, it doesn't work.
I then move to my ingress-controller freshly created from the link above. The logs I get are:
I don't really understand all the logs but it doesn't seem to display much about me pinging...
Do someone has any idea about what is really happening and if I have any way to maybe debug something? Because right now, based on the Internet, I am good to go, but in reality, nothing seems to work at all...
I'm using an Nginx ingress controller to direct traffic to a specific service, although right now there is only one. The service points to a Node.js app, and the html returned from the server is always 304 not modified. Does anyone have any ideas as to how I can return 200 and prevent cache?
Here's my Ingress settings:
apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
name: kludge-ingress
namespace: kludge
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: kludge.local
http:
paths:
- path: /
backend:
serviceName: kludge-front-end
servicePort: 80
Service:
apiVersion: v1
kind: Service
metadata:
name: kludge-front-end
namespace: kludge
spec:
type: NodePort
selector:
name: kludge-front-end
ports:
- name: http
protocol: TCP
nodePort: 30080
port: 80
targetPort: 3000
Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kludge-front-end
namespace: kludge
spec:
replicas: 3
selector:
matchLabels:
name: kludge-front-end
template:
metadata:
namespace: kludge
labels:
name: kludge-front-end
spec:
containers:
- name: kludge-front-end
image: gibson445/kludge-front-end
ports:
- containerPort: 3000
imagePullPolicy: Never
There's a pretty good chance you'd actually want configuration-snippet to add the header Cache-Control.
Example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myingress
namespace: mynamespace
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
add_header Cache-Control "max-age=0, no-cache, no-store, must-revalidate";