Sticky session for ASP.NET Core on Kubernetes deployment - nginx

I try to port an ASP.NET Core 1 application with Identity to Kubernetes. The login doesn't work and I got different errors like The anti-forgery token could not be decrypted. The problem is that I'm using a deployment with three replica sets so that further request were served by different pods that don't know about the anti-forgery token. Using replicas: 3 it works.
In the same question I found a sticky session documentation which seems a solution for my problem. The cookie name .AspNetCore.Identity.Application is from my browser tools.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myapp-k8s-test
annotations:
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: ".AspNetCore.Identity.Application"
spec:
replicas: 3
template:
metadata:
labels:
app: myapp-k8s
spec:
containers:
- name: myapp-app
image: myreg/myapp:0.1
ports:
- containerPort: 80
env:
- name: "ASPNETCORE_ENVIRONMENT"
value: "Production"
imagePullSecrets:
- name: registrypullsecret
This doesn't work, either with or without leading dot at the cookie name. I also tried adding the following annotations
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/session-cookie-hash: sha1
What is required to allow sticky sessions on Kubernetes with ASP.NET Core?

Found out that I made two logical mistakes:
Sticky sessions doesn't work this way
I assumed that Kubernetes will look into the cookie and create some mapping of cookie hashes to pods. But instead, another session is generated and append to our http header. nginx.ingress.kubernetes.io/session-cookie-name is only the name of those generated cookie. So per default, it's not required to change them.
Scope to the right object
The annotations must be present on the ingress, NOT the deployment (stupid c&p mistake)
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-k8s-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-hash: sha1
spec:
tls:
- hosts:
- myapp-k8s.local
rules:
- host: myapp-k8s.local
http:
paths:
- path: /
backend:
serviceName: myapp-svc
servicePort: 80
This works as expected.

Related

RabbitMQ management plugin: there is no template when using rewrite target from ingress nginx

Kind of a weird one.
I have a cluster of RabbitMQ running on K8, using the RabbitMQ Cluster Kubernetes Operator. Nothing fancy for the Yaml config file. The name of the RabbitMQ service is dev-rabbitmq. The port 15672 (default) is to have access to the Management UI. Port forwarding works fine.
I have set-up a Ingress Nginx service to be able to customize the url in my browser. This, in order to have only one load balancer and have different URLs for different services
There are 2 scenarios, one which works (but not what I'm looking for). The other I get the below error message being displayed (and no option to login) only for Firefox.
I've tried using Firefox, Edge and Chrome
What works
Ingress config
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-rabbitmq
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: nginx-dev
app.kubernetes.io/component: reverse-proxy
app.kubernetes.io/managed-by: helm
niiwaa.com/environment: development
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: rabbits
spec:
ingressClassName: nginx
rules:
- http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: dev-rabbitmq
port:
number: 15672
host: dev.hostname.com
Going to dev.hostname.com in my browser works as expected
If I change the path under spec.rules.http.paths.path to
path: /dev-rabbitmq
Then I get the error message in my browsers and no form to login.
I inspected the network console on both Edge and Firefox. On both cases, they send the request "GET http://dev.hostname.com/js/ejs-1.0.min.js"
Only when my path is set to "/" that it is successful. Otherwise, I get a 404 error.
The reason is that the resource is at http://dev.hostname.com/dev-rabbitmq/js/ejs-1.0.min.js.
Thanks to this question and answers I was able to resolve it.
In my Yaml file for the RabbitMQ cluster, I added an additional config like so
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
name: dev-rabbitmq
labels:
app.kubernetes.io/name: rabbitmq
app.kubernetes.io/instance: rabbitmq-dev
app.kubernetes.io/component: task-orchestrator
app.kubernetes.io/part-of: tasks
app.kubernetes.io/managed-by: rabbitmqclusters
niiwaa.com/environment: development
spec:
rabbitmq:
additionalConfig: |
management.path_prefix = /dev-rabbitmq
And I removed the annotation rewrite-target
annotations:
kubernetes.io/ingress.class: nginx
namespace: rabbits

How to setup an ingress controller for kubernetes?

Sorry I am new with Kubernetes and everything...
I have a java back-end in a clusterIP service and a front-end in a NodePort service. I try to make a request to the backend from the front (from the navigator) and it doesn't work.
I saw that I needed to setup an ingress crontroller in order to make it work, but each time I do a "minikube tunnel" and go to my localhost, I get a NGINX 404 error. And the address http://toto.virtualisation doesn't work too (like it doesn't exist).
Here is the setup of my front and my ingress controller in my yaml file :
# Front Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: front-end-deployment
spec:
selector:
matchLabels:
app: front-end
template:
metadata:
labels:
app: front-end
spec:
containers:
- name: front-end-container
image: oxasa/front-end-image:latest
---
# Front Service
apiVersion: v1
kind: Service
metadata:
name: front-end-service
spec:
ports:
- name: http
targetPort: 80
port: 80
type: NodePort
selector:
app: front-end
---
# Front Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: front-end-ingress
spec:
rules:
- host: toto.virtualisation
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: front-end-service
port:
number: 80
If you see anything that needs to be done to make it work...
Try adding
spec:
ingressClassName: nginx
To the Ingress resource to ensure nginx picks up the created ingress.
Also Ingress is not required for service to service Communication. You can use the Kubernetes internal DNS from your Frontend service.
You can make the Frontend access backend by using sth like {service-name}.{namespace}.svc.cluster.local

GKE Nginx Ingress Controller Oauth2 Proxy redirect

I am trying to add authentication to my cluster by using an oauth2-proxy. Locally I tested and is working as expected: When I go to the landing page it redirects me to an IP such as GitHub. After login, I am redirected to my page and everything is working as expected. For some odd reason, when porting from local to the cluster living in the google cloud, I am not getting redirect to GitHub for authentication.
I have two ingresses, one for the oauth2-proxy (ingress-oauth2-proxy) and another for all of the apps (ingress-apps). I added the Nginx annotations and still nothing.
Here are the Ingress YAML file used for the creation of the ingress rules
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
acme.cert-manager.io/http01-edit-in-place: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/auth-url: "https://auth.example.com/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://auth.exmaple.com/oauth2/start?rd=$escaped_request_uri"
cert-manager.io/cluster-isuer: letsecnrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: $/1
name: ingress-apps
namespace: default
spec:
rules:
- host: echo.example.com
http:
paths:
- path: /
backend:
serviceName: echo1
servicePort: 80
tls:
- hosts:
- echo.exmaple.com
secretName: echo-tls-cert
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-oauth2-proxy
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
acme.cert-manager.io/http01-edit-in-place: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: auth.exmaple.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- auth.exmaple.com
secretName: auth-tls-cert
I checked the logs of the OAuth proxy and when I go to echo.example.com nothing happens. If I make a request to auth.example.com/oauth2 I get redirected, as expected, to the IP login page which is GitHub in this case.
Am I missing something?
Note: I checked and both ingresses are being applied.
Ok, I figured it out: The problem was the Ingress controller used. I installed the Nginx controller from the repo helm.nginx.com/stable which is nginxinc and does not support the annotations. Therefore they were being ignored. To fix it I just used the kubernetes.github.io/ingress-nginx/deploy/#gce-gke. This is better explained here Nginx ingress controller authentication not working
Cheers ! :)

Kubernetes Nginx Ingress partial ssl termination

I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routingĀ¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.

Minikube Ingress Does not resolve but minikube IP does

I am running a simple pod with an image from local image registry in a minikube cluster on Windows 10. I am also running a simple nodeport service. The container is available when I try accessing it from the browser with <minikube_ip>:30080.
However, now I want to set an ingress controller because I want to set up a domain and not access it using the IP. The ingress works for something simple like a basic nginx pod, but does not work for this pod that I'm trying to use. I was previously using jwilder/nginx-proxy in docker-compose and it had some conf files that needed to be attached in the conf.d directory. However, since I am moving to Kubernetes, I thought to totally omit the conf files and the reverse proxy image.
Now after the hosts fie is updated, the domain is reachable via curl, the domain is also pingable, however, it simply cannot be reached on the browser.
pod-yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
io.kompose.service: api
name: api
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: api
strategy:
type: RollingUpdate
template:
metadata:
labels:
io.kompose.service: api
spec:
containers:
- env:
- name: DEV_PORT
value: "80"
image: localhost:5000/api:2.3
imagePullPolicy: "IfNotPresent"
name: api
resources: {}
restartPolicy: Always
serviceAccountName: ""
status: {}
Service.yaml
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: C:\Users\***kompose.exe
convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
io.kompose.service: api
name: api
spec:
selector:
io.kompose.service: api
type: NodePort
ports:
- name: "http"
port: 80
targetPort: 80
nodePort: 30080
Ingress.yaml
apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
name: tls-ingress
spec:
tls:
- secretName: oaky-tls
hosts:
- api.localhost
rules:
- host: api.localhost
http:
paths:
- path: /
backend:
serviceName: api
servicePort: 80
I have checked and the TLS secret is available, I am not understanding the issue here and would really appreciate some help.
Solved:
Chrome was overlooking the etc hosts file, so I did the following:
Switched to Firefox and instantly the URLs were working.
Added annotations to denote the class:
kubernetes.io/ingress.class: nginx
Added annotations to make sure requests are redirected to ssl
nginx.ingress.kubernetes.io/ssl-redirect: "true"

Resources