I am trying to add authentication to my cluster by using an oauth2-proxy. Locally I tested and is working as expected: When I go to the landing page it redirects me to an IP such as GitHub. After login, I am redirected to my page and everything is working as expected. For some odd reason, when porting from local to the cluster living in the google cloud, I am not getting redirect to GitHub for authentication.
I have two ingresses, one for the oauth2-proxy (ingress-oauth2-proxy) and another for all of the apps (ingress-apps). I added the Nginx annotations and still nothing.
Here are the Ingress YAML file used for the creation of the ingress rules
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
acme.cert-manager.io/http01-edit-in-place: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/auth-url: "https://auth.example.com/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://auth.exmaple.com/oauth2/start?rd=$escaped_request_uri"
cert-manager.io/cluster-isuer: letsecnrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: $/1
name: ingress-apps
namespace: default
spec:
rules:
- host: echo.example.com
http:
paths:
- path: /
backend:
serviceName: echo1
servicePort: 80
tls:
- hosts:
- echo.exmaple.com
secretName: echo-tls-cert
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-oauth2-proxy
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
acme.cert-manager.io/http01-edit-in-place: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: auth.exmaple.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- auth.exmaple.com
secretName: auth-tls-cert
I checked the logs of the OAuth proxy and when I go to echo.example.com nothing happens. If I make a request to auth.example.com/oauth2 I get redirected, as expected, to the IP login page which is GitHub in this case.
Am I missing something?
Note: I checked and both ingresses are being applied.
Ok, I figured it out: The problem was the Ingress controller used. I installed the Nginx controller from the repo helm.nginx.com/stable which is nginxinc and does not support the annotations. Therefore they were being ignored. To fix it I just used the kubernetes.github.io/ingress-nginx/deploy/#gce-gke. This is better explained here Nginx ingress controller authentication not working
Cheers ! :)
Related
I am trying to reach out to two services from my DNS.
Example:- myportal.com
myportal.com/app1
I have my SSO and SSL implementation on "myportal.com" and on the home page I have a button which calls a service "myportal.com/app1". I have tried multiple things but I am not able to configure both properly.
From below ingress config I am able to redirect to "myportal.com/app1" but its not working properly for "myportal.com". I have tried multiple thing including default backend and its not working for me. When I use default backend then "myportal.com" works but it stops redirecting to /app1.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myportal-ingress-test
namespace: appspace
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- myportal.com
secretName: secret
rules:
- host: myportal.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: portal-service
port:
number: 8097
- pathType: Prefix
path: /app1(/|$)(.*)
backend:
service:
name: app-ui-service
port:
number: 8087
I am fairly new to kubernetes and I am not sure if it's the correct way to implement such things. I am open to new implementation too if it's using ingress and routing to different services using same dns on different path.
Thanks in advance for the help!
I was able to resolve this issue. The redirect in an ingress file would work for all the paths, so I created a separate ingress for each path with individual redirects.
Ingress 1 - myportal.com (with SSO and SSL)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gcpportal-service-test
namespace: appspace
annotations:
nginx.ingress.kubernetes.io/x-forwarded-prefix: /myportal-service
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
ingressClassName: nginx
tls:
- hosts:
- myportal.com
secretName: secret
defaultBackend:
service:
name: myportal-service
port:
number: 80
rules:
- host: myportal.com
Ingress 2 - myportal.com/app1 (application deployed)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gcpportal-service-test-1
namespace: appspace
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- myportal.com
secretName: secret
rules:
- host: myportal.com
- pathType: Prefix
path: /app1(/|$)(.*)
backend:
service:
name: app1-ui-service
port:
number: 8087
This helped me to resolve this issue. I also verified that this way is also a standard way and is provided in one of the documentation of nginx.
https://kubernetes.github.io/ingress-nginx/user-guide/ingress-path-matching/
I use Axios in React and Django Rest Framework with dj-rest-auth. After migrating from GCP to Azure and removing the unmaintained django-rest-auth, I got some new CSRF issues.
Initially I removed django-rest-auth and created my own LoginView from Django.contrib.auth.views. Noticed that this also gave the CSRF error in development. So I added dj-rest-auth, which solved my issue locally. Pushed to the AKS, but there the cookie still does not appear. I'm suspecting my ingress to be the problem, which is able to set INGRESSCOOKIE for both my backend and frontend, but no CSRF.
I know there's a million tickets about this topic, my Django settings are fine, the set-cookie resopnse header is set. Also use the right axios settings to make sure that if the cookie is there, it's used for requests. The problem is really with the fact that the set-cookie is not coming through, as it is being created in Django.
I use an Nginx controller with TLS on a static IP with and have my ingress defined as follows:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: basic-ingress
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.allow-http: "true"
spec:
tls:
- hosts:
- XXXXX
secretName: tls-secret
rules:
- host: XXXXXXXX
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend
port:
number: 80
- path: /api/
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
The problems was indeed at the ingress, not so much with the settings, but with the routing. The routing used above, rewrites the targets to / for all requests. My backend actually has a page there, which is CSRF protected. It went to that page instead of the login page. Changing the ingress to below yielded better results.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: basic-ingress
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: /$1
kubernetes.io/ingress.allow-http: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- XXXXXXXXXX
secretName: tls-secret
rules:
- host: XXXXXX
http:
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: frontend
port:
number: 80
- path: /api/?(.*)
pathType: Prefix
backend:
service:
name: backend
port:
number: 8080
I have a kubernetes cluster with an application (deployment + ClusterIp service), nginx ingress controller, cert manager and letsencrypt issuer.
Here the service
apiVersion: v1
kind: Service
metadata:
name: myapp-service
namespace: mynamespace
spec:
selector:
app: myapp
ports:
- protocol: TCP
port: 80
targetPort: 80
type: ClusterIP
This is the the ingress yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
namespace: mynamespace
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt
spec:
tls:
- hosts:
- <myapp>.<myregion>.cloudapp.azure.com
secretName: tls-secret
rules:
- host: <myapp>.<myregion>.cloudapp.azure.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-service
port:
number: 80
It works correctly, responding to the url https://<myapp>.<myregion>.cloudapp.azure.com.
Now I need to change the path like in :
spec:
tls:
- hosts:
- <myapp>.<myregion>.cloudapp.azure.com
secretName: tls-secret
rules:
- host: <myapp>.<myregion>.cloudapp.azure.com
http:
paths:
- path: /sub
pathType: Prefix
backend:
service:
name: myapp-service
port:
number: 80
I would expect to browse my app at https://<myapp>.<myregion>.cloudapp.azure.com/sub.
Instead I get
This <myapp>.<myregion>.cloudapp.azure.com page can’t be found
What I am doing wrong?
I tried to find examples online, but couldn't find any that helped me understand what's wrong.
EDIT
What happens behind the scenes (dev tools) is:
The browser sends a request to /sub
The ingress routes to the correct service, rewriting the url to /
The application receives the request correctly
The application wants to redirect the browser to a login url (e.g. /login)
The browser receives a redirect (302) to /login and executes it
The ingress doesn't see the /sub in the redirect url, so it doesn't know what to do
I guess the redirect url should be /sub/login, not simply /login.
There should be an easy way to configure the ingress to fix this trivial issue. Can someone point me to the right direction?
I have no idea what's the problem for my case.
I deploy an Prometheus server on AKS(Azure's k8s) and want to expose the Prometheus web UI through ingress controller for the following config.
And I also refer this
https://coreos.com/operators/prometheus/docs/latest/user-guides/exposing-prometheus-and-alertmanager.html
# Prometheus
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: prometheus
namespace: monitoring
spec:
version: v2.13.1
replicas: 2
retention: 1d
serviceAccountName: prometheus
...
# Service
apiVersion: v1
kind: Service
metadata:
name: prometheus-service
namespace: monitoring
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 9090
selector:
app: prometheus
# Ingress
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: nginx
# nginx.ingress.kubernetes.io/ssl-redirect: "false"
# nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
# Just try another subpath make sure the nginx is work
# - backend:
# serviceName: aks-helloworld-one
# servicePort: 80
# path: /hello-world-one
- backend:
serviceName: prometheus-service
servicePort: 80
path: /prometheus
I have added another path in my test stage, the nginx work successfully for aks-helloworld-one.
However, it not work for Prometheus server, I always got "404 page not found" in return.
Does anyone know how to solve this problem?
I think I know where the problem is. Actually my previous comment was wrong:
If it's available under /, you don't need any rewrites in your ingress
and it should work straight away.
Well... actually it won't for one simple reason. When you try to access your Prometheus UI via ingress, you use <ingress ip>/prometheus URL. Choosing /prometheus path redirects you correctly to the appropriate backend Service:
- backend:
serviceName: prometheus-service
servicePort: 80
path: /prometheus
But the problem occurs because the /prometheus path (that you need to be redirected to the prometheus-service and eventually to one of the Pods, exposed by this Service) gets forwarded to the target Pod serving the actual content.
You get 404 page not found error message because the http request that gets to the target webserver is claiming for the content of /prometheus directory instead of / from which it is actually served.
So if you change your ingress to something like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- backend:
serviceName: prometheus-service
servicePort: 80
path: /
most probably everything would work as expected and you'll get the Prometheus UI website rather than 404 Not found.
Well, although it may work, it's good only for debugging purposes as no one wants to use ingress just to be able to expose something under root path.
The following ingress definition should resolve your problem (Yes, rewrites are necessary in this scenario!):
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- backend:
serviceName: prometheus-service
servicePort: 80
path: /prometheus(/|$)(.*)
Rewrite that was used above will ensure that the original access path /prometheus gets rewritten to / before reaching the target Pod.
I am deploying the cluster in a AWS, I did this configuration with Helm Charts and it worked for me.
Ingress: Notice nginx.ingress.kubernetes.io/rewrite-target: /$2 and path: /prometheus(/|$)(.*) in this file as mentioned in other answers.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/proxy-read-timeout: "12h"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
cert-manager.io/acme-challenge-type: "http01"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
tls:
- hosts:
- myhost.cl
secretName: mysecret-tls
rules:
- host: myhost.cl
http:
paths:
- path: /prometheus(/|$)(.*)
pathType: Prefix
backend:
service:
name: monitoring-kube-prometheus-prometheus
port:
number: 9090
Prometheus: There is a label inside spec called externalUrl that makes you redirect whatever you want when you access in browser.
# Source: kube-prometheus-stack/templates/prometheus/prometheus.yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: monitoring-kube-prometheus-prometheus
namespace: monitoring
labels:
app: kube-prometheus-stack-prometheus
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/instance: monitoring
app.kubernetes.io/version: "19.0.2"
app.kubernetes.io/part-of: kube-prometheus-stack
chart: kube-prometheus-stack-19.0.2
release: "monitoring"
heritage: "Helm"
prometheus: devops
spec:
# ... Your data here
externalUrl: http://myhost.cl/prometheus
Also be careful to not expose your prometheus instance and at least use an authentication method like basic auth, here's something that can help you with that: https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
I think issue is with your rewrite-target & path, can you try with a sub-domain, rather than path to confirm? I have the following Prometheus Ingress working on Subdomain
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx-ingress
name: prometheus-monitoring
namespace: monitoring
spec:
backend:
serviceName: prometheus
servicePort: 9090 #As my service is listening on 9090
rules:
- host: prometheus-monitoring.DOMAIN
http:
paths:
- backend:
serviceName: prometheus
servicePort: 9090 #As my service is listening on 9090
path: /
And below is my service manifest
apiVersion: v1
kind: Service
metadata:
labels:
app: prometheus
chart: prometheus-operator-5.11.0
heritage: Tiller
name: prometheus
namespace: monitoring
spec:
ports:
- name: web
port: 9090
protocol: TCP
targetPort: 9090
selector:
app: prometheus
prometheus: k8s
sessionAffinity: None
type: ClusterIP
I have one service called "workspace-service-b6" which is running on port 5000, See the below ingress file. Now I want to serve the static content on the same service (workspace-service-b6) by adding the path route.
Example:- Service is working on https://workspace-b6.dev.example.com
Now if the user adds "/workspace/v2/ "at the end of the URL.
Like this:- https://workspace-b6.dev.example.com/workspace/v2/ it will redirect to s3 bucket "https://s3.console/buckets/xyz/abc/build" where my static content is available.
My Ingress file :-
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: b6-ingress
namespace: b6
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/tls-acme: "true"
spec:
tls:
- hosts:
- workspace-b6.dev.example.com
secretName: xyz-crt
rules:
- host: workspace-b6.dev.example.com
http:
paths:
- backend:
serviceName: workspace-service-b6
service port: 5000
While it’s kind of possible, the real answer is “don’t”. The ingress system is just a proxy, set up separate pods for content.