I am in the process of changing from an Azure webservice to azure kubernetes to host an api. I have the solution working with nginx and oauth2_proxy and azure active directory. However the solution requires a cookie to function.
As this is an api and the external security will be managed by an AWS API Gateway with a custom authoriser. I would like for the API Gateway to authenticate using a bearer token only and not require a cookie.
I have my solution working and have been so far testing form postman. In postman I have the bearer token but cannot find a way to access without the cookie.
My application presently runs via aws api gateway and an azure app service with azure active directory. The aws api gateway custom authoriser does not require a cookie in this case.
I have the following configuration
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: oauth2-proxy
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: mydomain.com
http:
paths:
- path: /oauth2
backend:
serviceName: oauth2-proxy
servicePort: 4180
tls:
- hosts:
- mydomain.com
secretName: tls-secret
------
# oauth2_proxy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: oauth2-proxy
spec:
replicas: 1
selector:
matchLabels:
app: oauth2-proxy
template:
metadata:
labels:
app: oauth2-proxy
spec:
containers:
- env:
- name: OAUTH2_PROXY_PROVIDER
value: azure
- name: OAUTH2_PROXY_AZURE_TENANT
value: mytennantid
- name: OAUTH2_PROXY_CLIENT_ID
value: my clientid
- name: OAUTH2_PROXY_CLIENT_SECRET
value: my client secret
- name: OAUTH2_PROXY_COOKIE_SECRET
value: my cookie secret
- name: OAUTH2_PROXY_HTTP_ADDRESS
value: "0.0.0.0:4180"
- name: OAUTH2_PROXY_UPSTREAM
value: "file:///dev/null"
image: machinedata/oauth2_proxy:latest
imagePullPolicy: IfNotPresent
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
spec:
ports:
- name: http
port: 4180
protocol: TCP
targetPort: 4180
selector:
app: oauth2-proxy
-----
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
spec:
tls:
- hosts:
- mydomain.com
secretName: tls-secret
rules:
- host: mydomain.com
http:
paths:
- backend:
serviceName: mayapp
servicePort: 80
I would like to change this configuration so a cookie is no longer required. If this is not possible is there another way to achieve the same outcome?
Just drop the oauth part on kubernetes and make API Gateway validate the requests, it has the ability to do exactly what you need. You can secure your kubernetes to only accept requests from the API Gateway, so you don't need to protect your endpoint from other calls.
Related
in my k8s system I have a nginx ingress controller as LoadBalancer and accessing it to ddns adress like hedehodo.ddns.net and this triggering to forward web traffic to another nginx port.
Now I deployed another nginx which works on node.js app but I cannot forward nginx ingress controller for any request to port 3000 to go another nginx
here is the nginx ingress controller yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
namespace: default
spec:
rules:
- host: hedehodo.ddns.net
http:
paths:
- path: /
backend:
serviceName: my-nginx
servicePort: 80
- path: /
backend:
serviceName: helloapp-deployment
servicePort: 3000
helloapp deployment works a Loadbalancer and I can access it from IP:3000
could any body help me?
Each host cannot share multiple duplicate paths, so in your example, the request to host: hedehodo.ddns.net will always map to the first service listed: my-nginx:80.
To use another service, you have to specify a different path. That path can use any service that you want. Your ingress should always point to a service, and that service can point to a deployment.
You should also use HTTPS by default for your ingress.
Ingress example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
rules:
- host: my.example.net
http:
paths:
- path: /
backend:
serviceName: my-nginx
servicePort: 80
- path: /hello
backend:
serviceName: helloapp-svc
servicePort: 3000
Service example:
---
apiVersion: v1
kind: Service
metadata:
name: helloapp-svc
spec:
ports:
- port: 3000
name: app
protocol: TCP
targetPort: 3000
selector:
app: helloapp
type: NodePort
Deployment example:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloapp
labels:
app: helloapp
spec:
replicas: 1
selector:
matchLabels:
app: helloapp
template:
metadata:
labels:
app: helloapp
spec:
containers:
- name: node
image: my-node-img:v1
ports:
- name: web
containerPort: 3000
You can't have the same "path: /" for the same host. Change the path to a different one for your the new service.
I have created a brand new K8S cluster
I have created the Ingress nginx controller.
The controller created a namespace with all of the required Pods, Svcs and etc.
I have created an Ingress object that routes the traffic to a Deployment service with TLS enabled.
I have created a cluster issuer object.
When inspecting the kubectl describe cert everything okay and ready.
When inspecting the kubectl describe clusterissuer, as well.
When doing curl https://example.com/ it returns the following error:
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Without SSL, the access is enabled from outside and works properly, when adding back the SSL configuration in the Ingress object, it fails again.
ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- k8s-poc.example.com
secretName: echo-tls
rules:
- host: k8s-poc.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test-svc
port:
number: 3333
test-depl.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-depl
labels:
app: test
spec:
replicas: 1
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: test
image: mydockeruser/test:42
ports:
- containerPort: 3333
imagePullSecrets:
- name: docker-regcred
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
name: test-svc
spec:
selector:
app: test
ports:
- name: http
protocol: TCP
port: 3333
targetPort: 3333
prod-issuer.yaml:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
namespace: cert-manager
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: my#email.com
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-prod
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
i am running an Nginx ingress controller and wanted to allow only few path for users to connect and rest all I wanted to block or provide an 403 error. how can i do that?
I only wanted users to allow to connect /example and rest all should be blocked.
kind: Ingress
metadata:
name: ingress1
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: ingress.example.com
http:
paths:
- path: /example
backend:
serviceName: ingress-svc
servicePort: 80
Can i add a nginx server-snippet?
location path {
"if the path is not matching then deny"
deny all;
}```
Make a custom backend using below
apiVersion: apps/v1
kind: Deployment
metadata:
name: custom-http-backend
spec:
selector:
matchLabels:
app: custom-http-backend
template:
metadata:
labels:
app: custom-http-backend
spec:
containers:
- name: custom-http-backend
image: inanimate/echo-server
ports:
- name: http
containerPort: 8080
imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
name: custom-http-backend
spec:
selector:
app: custom-http-backend
ports:
- protocol: TCP
port: 80
targetPort: 8080
Then in your ingress add this rule
- host: ingress.example.com
http:
paths:
- path: /
backend:
serviceName: custom-http-backend
servicePort: 80
Additionally to what #Tarun Khosla mentioned which is correct, there is another stackoverflow question with examples which might be helpful. I am posting this as a community wiki answer for better visibility for the community, feel free to expand on it.
There are 2 examples provided by #Nick Rak
I’ve faced the same issue and found the solution on github.
To achieve your goal, you need to create two Ingresses first by default without any restriction:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-test
spec:
rules:
- host: host.host.com
http:
paths:
- path: /service-mapping
backend:
serviceName: /service-mapping
servicePort: 9042
Then, create a secret for auth as described in the doc:
Creating the htpasswd
$ htpasswd -c auth foo
New password: <bar>
New password:
Re-type new password:
Adding password for user foo
Creating the secret:
$ kubectl create secret generic basic-auth --from-file=auth
secret "basic-auth" created
Second Ingress with auth for paths which you need to restrict:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropiate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
spec:
rules:
- host: host.host.com
http:
paths:
- path: /admin
backend:
serviceName: service_name
servicePort: 80
According to sedooe answer, his solution may have some issues.
and #sedooe
You can use server-snippet annotation. This seems like exactly what you want to achieve.
I have an issue I can't figure out. I have setup Nginx Ingress Controller on my managed k8s cluster. I'm trying to reach an SSL enabled pod behind and it does not work. I have 404 not found from Nginx and the certificate which is presented is the Nginx one. I have deployed the controller using their github repo and the default files following their doc.
I have setup a clear http pod for purpose tests and it works. It seems to be related to ssl.
I have tried many things to no avail. How can I reach an SSL pod behind nginx ?
Here's the Deployment + service (for the https one) resource I have setup :
apiVersion: apps/v1
kind: Deployment
metadata:
name: moulip-https
spec:
selector:
matchLabels:
app: moulip-https
replicas: 2
template:
metadata:
labels:
app: moulip-https
spec:
containers:
- name: "wabam"
image: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
ports:
- containerPort: 443
imagePullSecrets:
- name: regcrd
---
apiVersion: v1
kind: Service
metadata:
name: https-svc
labels:
app: moulip-https
spec:
ports:
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: moulip-https
and my Ingress :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: default
spec:
rules:
- host: https.moulip.lan
http:
paths:
- backend:
serviceName: https-svc
servicePort: 443
- host: test.moulip.lan
http:
paths:
- backend:
serviceName: hostname-svc
servicePort: 80
Many thanks for any guidance you could provide me with.
You are missing tls configuration in the ingress. follow sample below
apiVersion: v1
kind: Secret
metadata:
name: testsecret-tls
namespace: default
data:
tls.crt: base64 encoded cert
tls.key: base64 encoded key
type: kubernetes.io/tls
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: tls-example-ingress
spec:
tls:
- hosts:
- sslexample.foo.com
secretName: testsecret-tls
rules:
- host: sslexample.foo.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80
Cloud: Google Cloud Platform.
I have the following configuration
kind: Deployment
apiVersion: apps/v1
metadata:
name: api
spec:
replicas: 2
selector:
matchLabels:
run: api
template:
metadata:
labels:
run: api
spec:
containers:
- name: api
image: gcr.io/*************/api
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: /_ah/health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
---
kind: Service
apiVersion: v1
metadata:
name: api
spec:
selector:
run: api
type: NodePort
ports:
- protocol: TCP
port: 8080
targetPort: 8080
---
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: main-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /api/*
backend:
serviceName: api
servicePort: 8080
All set. GKE saying that all deployments are okay, pods number are met and main ingress with nginx-ingress-controllers are set as well. But I'm not able to reach any of the services. Even application specific 404. Nothing. Like, it's not resolved at all.
Another related question I see to entrypoints. The first one through main-ingress. It created it's own LoadBalancer with own IP address. The second one address from nginx-ingress-controller. The second one is at least returning 404 from default backend, but also not pointing to expected api service.