websocket connection failed after establishing https in google ingress controller - nginx

I have deployed an application in kubernetes which is served by Google Ingress Controller (Service as ELB). The application is working fine. But the moment I am applying https related configuration, the https is coming but websocket fails.
Below is the service file and configmap
for http:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app: ingress-nginx
annotations:
# Enable PROXY protocol
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
# Increase the ELB idle timeout to avoid issues with WebSockets or Server-Sent Events.
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
spec:
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---------------------------------------------------------------------------------------------------
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
use-proxy-protocol: "true"
for https:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-east-1:2xxxxxxxxxxxxxxxxxxx56:certificate/3fxxxxxxxxxxxxxxxxxxxxxxxxxx80"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
# Increase the ELB idle timeout to avoid issues with WebSockets or Server-Sent Events.
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
spec:
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: http
------------------------------------------------------------------------------------------
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
use-proxy-protocol: "false"
Am I missing any annotations or data in configmap ? Pls help me out

I think the problem is the annotation:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
The backend-protocol in ELBs must be TCP for websocket connections.
Also, I see you're using Nginx Ingress Controller, maybe you want to set these variables in the config
proxy-read-timeout: "3600"
proxy-send-timeout: "3600"
To avoid connection closings.

Related

400: Bad Request blog page via http/https SSL-enabled k3s deployment

Using nginx-ingress controller and metallb for my loadbalancer in my k3s raspberry pi cluster. Trying to access my blog site but I get white page with 400: Bad Request.
I'm using Cloudflare to managed my domain and SSL/TLS mode is on "Full". Created an A name "Blog" and pointed the content to my public external IP. I opened the Loadbalancer IP address on my router exposing 80 and 433. What am I missing. I've been pulling my hair with his issues for days now. Here's the example of my k3s entire deployment
apiVersion: v1
kind: Service
metadata:
namespace: nginx
name: nginx-web
labels:
app: nginx-web
spec:
ports:
# the port that this service should serve on
- port: 8000
targetPort: 80
protocol: TCP
selector:
app: nginx-web
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
replicas: 1
selector:
matchLabels:
app: nginx-web
template:
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
containers:
- name: nginx-web
image: nginx:latest
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
labels:
app: nginx-web
namespace: nginx
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- blog.example.com
secretName: blog-example-com-tls
rules:
- host: blog.example.com
http:
paths:
- backend:
service:
name: nginx-web
port:
number: 80
path: /
pathType: Prefix

i want to change my wordpress pod domain name

I hope you all are doing great today ,
here is my situation :
i have 2 wordpress websites (identical )
--1st one is an App Service wordpress in azure with a domain name eg:https://wordpress.azurewebsites.net
--the 2nd one is in aks cluster as a pod with a load balancer that expose it to the internet with a public ip
what i want to do :
i want to take the domain name from the app service and give it to the aks pod
what did i do :
i changed from the dashboard the domain name and changed the load balancer public ip adress
and it didn't work now i can't access the dashboard from the load balancer ip adress either
im new in kubernetes i hope someone can guide me to the right direction on how to do it
Seems like you are missing an ingress controller. You could for example install ingress-nginx and expose the ingress with this service config:
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
loadBalancerIP: 53.1.1.1
ports:
- name: https
port: 443
protocol: TCP
targetPort: https
appProtocol: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
You can now create a service for your app:
apiVersion: v1
kind: Service
metadata:
name: app_service
namespace: app
spec:
type: ClusterIP
ports:
- name: service
port: 80
selector:
app: yoour_app
Then you can expose yoour app with an ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app_ingress
namespace: app
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- wordpress.azurewebsites.net
rules:
- host: wordpress.azurewebsites.net
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app_service
port:
number: 80

yaml configuration for Istio and gRPC

I am working on a POC for Istio + gRPC, the Istio version is 1.6, but I could not see any gRPC traffic to my pods.
I suspect my Istio Gateway or VirtualService miss something, but I could not figure out what's wrong here? Could anybody help review my yaml file and correct me what's missing or wrong in my yaml?
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: syslogserver
name: syslogserver
namespace: mynamespace
spec:
selector:
matchLabels:
app: syslogserver
replicas: 1
template:
metadata:
labels:
app: syslogserver
spec:
containers:
- name: syslogserver
image: docker.io/grpc-syslog:latest
imagePullPolicy: Always
ports:
- containerPort: 5555
imagePullSecrets:
- name: pull-image-credential
---
apiVersion: v1
kind: Service
metadata:
name: syslogserver
namespace: mynamespace
labels:
app: syslogserver
spec:
selector:
app: syslogserver
ports:
- name: grpc
port: 6666
protocol: TCP
targetPort: 5555
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: xyz-ingress-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 7777
name: http2
protocol: HTTP2
hosts:
- "*"
---
apiVersion: v1
kind: Service
metadata:
name: xyz-istio-ingressgateway
namespace: istio-system
labels:
app: xyz-istio-ingressgateway
spec:
selector:
app: istio-ingressgateway
istio: ingressgateway
type: NodePort
ports:
- protocol: TCP
nodePort: 32555
port: 7777
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: xyz-ingress-gateway-virtualservice
namespace: istio-system
spec:
hosts:
- "*"
gateways:
- xyz-ingress-gateway
#tls:
http:
- match:
- port: 7777
route:
- destination:
host: syslogserver.mynamespace.svc.cluster.local
port:
number: 6666
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: xyz-destinationrule
namespace: istio-system
spec:
host: syslogserver.mynamespace.svc.cluster.local
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
Please give your guidance, thanks.
From what I see the service name: xyz-istio-ingressgateway should be deleted, as that's not how you communicate when using Istio.
Instead you should use istio ingress gateway, combined with a gateway, virtual service and destination rule.
If you've choosen port number 7777 on your gateway, you have to open this port on istio ingress gateway, there are few ways to do that in this stackoverflow question. There are the default istio ingress gateway values.
After you configure the port you can use kubectl get svc istio-ingressgateway -n istio-system to get the istio ingress gateway external IP.
If the external IP value is set, your environment has an external load balancer that you can use for the ingress gateway. If the EXTERNAL-IP value is pending, then your environment does not provide an external load balancer for the ingress gateway. In this case, you can access the gateway using the service’s node port.
The rest of your configuration looks fine for me. Just a reminder about injecting a sidecar proxy to your pods.

Configuring Static IP address with Ingress Nginx Sticky Session on Azure Kubernetes

I am trying to configure an additional layer of Sticky Session to my current Kubernetes architecture. Instead of routing every request through the main LoadBalancer service, I want to route the requests through an upper layer of nginx sticky session. I am following the guide on https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/
I am using Azure Cloud for my cluster deployment. Previously, using a Service with LoadBalancer type would automatically generate an external IP address for users to connect to my cluster. Now I need to configure the static IP address for my users to connect to, with the nginx ingress in place. How can I do so? I followed the guide here - https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/static-ip but the external address of the Ingress is still empty!!
What did I do wrongly?
# nginx-sticky-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-lb
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Local
type: LoadBalancer
ports:
- port: 80
name: http
targetPort: 80
- port: 443
name: https
targetPort: 443
selector:
# Selects nginx-ingress-controller pods
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
# nginx-sticky-controller.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
containers:
- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.31.0
name: nginx-ingress-controller
ports:
- containerPort: 80
hostPort: 80
- containerPort: 443
hostPort: 443
resources:
limits:
cpu: 0.5
memory: "0.5Gi"
requests:
cpu: 0.5
memory: "0.5Gi"
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/nginx-ingress-lb
# nginx-sticky-server.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-nginx
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "nginx-sticky-server"
nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/affinity-mode: persistent
nginx.ingress.kubernetes.io/session-cookie-hash: sha1
spec:
rules:
- http:
paths:
- backend:
# This assumes http-svc exists and routes to healthy endpoints.
serviceName: my-own-service-master
servicePort: http
Ok I got it working. I think the difference lies in the cloud provider you are using, and for Azure Cloud, you should follow their documentation and their way of implementing ingress controller in the Kubernetes cluster.
Link over here for deploying the ingress controller. Their way of creating the public IP address within the Kubernetes cluster and linking it up with the ingress controller works. I can confirm as of now, the time of writing.
Once I am done deploying the steps in the link above, I can apply the ingress .yaml file as usual i.e. kubectl apply -f nginx-sticky-server.yaml to set up the nginx sticky session. IF the service name and service port stated in your ingress .yaml file is correct, the nginx ingress controller should redirect your user requests to the correct service.

rpc error: code = Unknown desc = Moved Permanently: HTTP status code 301

I have GRPC service written in go and i need to deploy the service on top of AWS-EKS, we are using nginx-ingress and cloudflare to point to our cluster gateway (nginx).
but when i tried to deploy the service and test it using this command grpcurl grpc.fd-admin.com:443 list
i always get the following error:
Failed to list services: rpc error: code = Unknown desc = Moved Permanently: HTTP status code 301; transport: missing content-type field
And this is what i did for kubernetes resources:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: grpc
labels:
k8s-app: grpc
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: grpc
spec:
containers:
- name: grpc
image: quay.io/kubernetes-ingress-controller/grpc-fortune-teller:0.1
ports:
- containerPort: 50051
name: grpc
---
apiVersion: v1
kind: Service
metadata:
name: grpc
namespace: grpc
spec:
selector:
k8s-app: grpc
ports:
- port: 50051
targetPort: 50051
protocol: TCP
name: grpc
type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: grpc
namespace: grpc
spec:
rules:
- host: grpc.fd-admin.com
http:
paths:
- backend:
serviceName: grpc
servicePort: grpc
tls:
- secretName: grpc
hosts:
- grpc.fd-admin.com
So can any one explain why i got this error or what is the reasons may cause this kind of error ?
Try below the annotations of nginx ingress,
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/grpc-backend-for-port: "grpc"
name: grpc

Resources