I'm new to K8s and currently following a course that is using K8s.I am using minikube in my local machine. I'm stuck on the part in which I need to directly communicate with ingress-nginx which happens to be in a separate namespace.
Here's some information regarding my setup.
✗ client (main) ✗ kubectl get namespace
NAME STATUS AGE
default Active 25d
ingress-nginx Active 21d
kube-node-lease Active 25d
kube-public Active 25d
kube-system Active 25d
Service
✔︎ client (main) ✗ kubectl get services -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.105.88.90 <none> 80:30306/TCP,443:31814/TCP 21d
ingress-nginx-controller-admission ClusterIP 10.101.75.17 <none> 443/TCP 21d
My ingress-srv.yaml file
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
labels:
name: ingress-service
spec:
rules:
- host: ticketing.dev
http:
paths:
- pathType: Prefix
path: "/api/users/(.*)"
backend:
service:
name: auth-srv
port:
number: 3000
- pathType: Prefix
path: /?(.*)
backend:
service:
name: client-srv
port:
number: 3000
I looked for some information over github and to do cross communication what I need to do is doing http://<service>.<namespace>.svc.cluster.local:<port>/. Based on that info, I came up with
http://ingress-nginx-controller.ingress-nginx.svc.cluster.local and test it using curl.
✔︎ client (main) ✗ curl http://ingress-nginx-controller.ingress-nginx.svc.cluster.local
curl: (6) Could not resolve host: ingress-nginx-controller.ingress-nginx.svc.cluster.local
I'm following a course called Microservices with Node JS and React
Hope for your answers and insights.
Thank you.
If you ingress controller in Ingress-nginx namespace it perfect or also fine it's any other namespace.
You have create the ingress object in that namespace where your application running, regardless of where you controller running.
Ingress will find the controller or communicate with it across namespace automatically using the class that you have specified in the ingress object.
In simple words
If your app1 running in namespace application just create one ingress in application namespace pointing to app1 service and you are done.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
labels:
name: ingress-service
spec:
rules:
- host: ticketing.dev
http:
paths:
- pathType: Prefix
path: "/api/users/(.*)"
backend:
service:
name: auth-srv
port:
number: 3000
- pathType: Prefix
path: /?(.*)
backend:
service:
name: client-srv
port:
number: 3000
Ingress Controller(simple Pod/Deployment along with a Service that can be used to utilize routing and proxying. Based on Nginx example);
Ingress rules(a separate Kubernetes resource with kind: Ingress. Will only take effect if Ingress Controller is already existing and class is mentioned in ingress)
Ingress Controller can be deployed in any namespace, usually deployed in a namespace separate from your app services like you have done in ingress-nginx. It can out-of-the-box see Ingress rules in all namespaces in the cluster and will pick them up.
The Ingress rules, however, must reside in the namespace where the app that they configure reside.
Related
I want to install nginx-controller in my Kubernetes cluster. I setup my master node at one server, and worker node at another server. I am using Ubuntu 20.04.
I followed the link (https://github.com/kubernetes/ingress-nginx/blob/main/deploy/static/provider/cloud/1.23/deploy.yaml) and use 'kubectl apply -f file_name.yaml' to install the controller.
The controller pod is running:
NAME READY STATUS RESTARTS AGE
ingress-nginx-controller-c57bb9674-p2z9d 1/1 Running 0 70s
Now I want to create an Ingress resource. I used this yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-hello
namespace: ingress-nginx
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /hello
pathType: Exact
backend:
service:
name: ingress-hello
port:
number: 80
However, when I applied this yaml file, and use 'kubectl get ingress -n ingress-nginx', I saw:
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-hello <none> * 80 24s
I noticed that the address for this Ingress resource is empty.
I am just wondering is it possible to assign an IP address to it? Any method/ setting to assign the address?
Thanks.
You can access to your service by http://localhost:80/hello, and if you wanan specify a custom host you need to modify your ingnx file.
This is an example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-hello
namespace: ingress-nginx
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: your_host
http:
paths:
- path: /hello
pathType: Exact
backend:
service:
name: ingress-hello
port:
number: 80
and you need to open your hosts files in your /system32/etc/hosts directory and add your customized host, and then your service will be accessible through
http://your_host:80/hello
Having trouble setting up external authentication for a web application behind nginx ingress. When i try to access the URL https://site.example.com from external i get no redirection to Github login, and direct access to web application happens.
Running Pods for my environment:
NAME READY STATUS
nginx-ingress-68df4dfc4f-wpj5t 1/1 Running
oauth2-proxy-6675d4b57c-cspw8 1/1 Running
web-deployment-7d4bd85b46-blxb8 1/1 Running
web-deployment-7d4bd85b46-nqjgl 1/1 Running
Active Services:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
nginx-ingress LoadBalancer 10.96.156.157 192.168.1.82 80:31613/TCP,443:32437/TCP
oauth2-proxy ClusterIP 10.100.101.251 <none> 4180/TCP
web-service ClusterIP 10.108.237.188 <none> 8080/TCP
Two Ingress resources:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
namespace: nginx-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$request_uri"
labels:
app: webapp
spec:
rules:
- host: site.example.com
http:
paths:
- path: /
backend:
serviceName: web-service
servicePort: 8080
tls:
- hosts:
- site.example.com
secretName: example-tls
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: oauth2-proxy
namespace: nginx-ingress
annotations:
kubernetes.io/ingress.class: nginx
labels:
app: oauth2-proxy
spec:
rules:
- host: site.example.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- site.example.com
secretName: example-tls
Ingress output:
NAME CLASS HOSTS ADDRESS PORTS
ingress <none> site.example.com 192.168.1.82 80, 443
oauth2-proxy <none> site.example.com 80, 443
I see these errors in Ingress oauth2-proxy events:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Rejected 54m nginx-ingress-controller All hosts are taken by other resources
Oauth2-proxy built from deployment here with Client ID, Client Secret and SECRET according to the OAuth app created in my Github account.
No logs found in oauth2-proxy logs, i suppose because it's not invoked in the process.
UPDATE:
This question was incomplete, i forgot to mention the NGINX image empolyed (NGINX 1.9.0 from installation guide).
Changing the image with the below:
NGINX Ingress controller
Release: v0.41.2
Build: d8a93551e6e5798fc4af3eb910cef62ecddc8938
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.19.4
the error disappears. In brief both Ingress configuration, the one from my question and the other from the answer are working.
In your configuration, you are using 2 Ingress. As you described you oauth2-proxy Ingress, in Event section you can find information:
All hosts are taken by other resources
Issue you have encounter here is called Host Collisions. It occured as in your both Ingress you have used:
spec:
rules:
- host: site.example.com
In that kind of situation, Ingress is using default algorith called Choosing the Winner.
If multiple resources contend for the same host, the Ingress Controller will pick the winner based on the creationTimestamp of the resources: the oldest resource will win.
As fast solution for your issue is to create one Ingress with 2 paths.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
namespace: nginx-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$request_uri"
spec:
rules:
- host: site.example.com
http:
paths:
- path: /
backend:
serviceName: web-service
servicePort: 8080
- path: /oauth2
backend:
serviceName: oauth2-proxy
servicePort: 4180
tls:
- hosts:
- site.example.com
secretName: example-tls
Another way to resolve this issue is to use Merging Configuration for the Same Host, however it shouldn't be applied in this scenario.
As last thing, you can follow official Nginx Ingress tutorial - External OAUTH Authentication
I'm a k8s newbie and I am trying to expose a port from the cluster into the local network.
I've tried to do it with metallb config layer2 + load balancer controller and this runs ok.
I have set up a 3-node environment with kubespray.
((192.168.0.1[5,6,7]))
However, I'm trying to expose an api with NodePort and NGINX-Ingress. The nodeport api service is running ok (i can make successfull requests via NODE_IP:NODE_nodeport). But if I try this ingress configuration it justs keeps telling me "connection refused":
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-api
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: "k8s.customhostname.com" # solves to 192.168.0.17 which has a running pod with the api.
http:
paths:
- path: /test
pathType: Prefix
backend:
service:
name: svc-api
port:
number: 8080
Then i check the services:
Name: svc-smouapimapes
Namespace: smouapi
Labels: app=apimapes
Annotations: <none>
Selector: app=apimapes
Type: ClusterIP
IP: 10.233.26.225
Port: springboot 8080/TCP
TargetPort: 8080/TCP
Endpoints: 10.233.90.4:8080,10.233.92.8:8080
Session Affinity: None
Events: <none>
And then check the ingress:
Name: ingress-smouapimapes
Namespace: smouapi
Address: 192.168.0.17
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
k8s.nexusgeografics.com
/test svc-smouapimapes:8080 10.233.90.4:8080,10.233.92.8:8080)
Annotations: nginx.ingress.kubernetes.io/ssl-redirect: false
Events: <none>
Whenever i call :
curl -I http://k8s.nexusgeografics.com/test
# CONNECTION REFUSED
What am I doing wrong?
Thanks
Try adding the following in the Nginx config ingress-smouapimapes.
Add annotation :
nginx.ingress.kubernetes.io/rewrite-target: /$2
And instead of this path: /test add path: /test(/|$)(.*)
Enin, you should configure the domain k8s.nexusgeografics.com to be resolved to the ip that providing the nginx-ingress service, thus, you can access your service through nginx-ingress.
I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routing¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.
I'm a Kubernetes amateur trying to use NGINX ingress controller on GKE. I'm following this google cloud documentation to setup NGINX Ingress for my services, but, I'm facing issues in accessing the NGINX locations.
What's working?
Ingress-Controller deployment using Helm (RBAC enabled)
ClusterIP service deployments
What's not working?
Ingress resource to expose multiple ClusterIP services using unique paths (fanout routing)
K8S Services
[msekar#ebs kube-base]$ kubectl get services -n payment-gateway-7682352
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-controller LoadBalancer 10.35.241.255 35.188.161.171 80:31918/TCP,443:31360/TCP 6h
nginx-ingress-default-backend ClusterIP 10.35.251.5 <none> 80/TCP 6h
payment-gateway-dev ClusterIP 10.35.254.167 <none> 5000/TCP 6h
payment-gateway-qa ClusterIP 10.35.253.94 <none> 5000/TCP 6h
K8S Ingress
[msekar#ebs kube-base]$ kubectl get ing -n payment-gateway-7682352
NAME HOSTS ADDRESS PORTS AGE
pgw-nginx-ingress * 104.198.78.169 80 6h
[msekar#ebs kube-base]$ kubectl describe ing pgw-nginx-ingress -n payment-gateway-7682352
Name: pgw-nginx-ingress
Namespace: payment-gateway-7682352
Address: 104.198.78.169
Default backend: default-http-backend:80 (10.32.1.4:8080)
Rules:
Host Path Backends
---- ---- --------
*
/dev/ payment-gateway-dev:5000 (<none>)
/qa/ payment-gateway-qa:5000 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"pgw-nginx-ingress","namespace":"payment-gateway-7682352"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"payment-gateway-dev","servicePort":5000},"path":"/dev/"},{"backend":{"serviceName":"payment-gateway-qa","servicePort":5000},"path":"/qa/"}]}}]}}
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: false
Events: <none>
Last applied configuration in the annotations (ingress description output) shows the ingress resource manifest. But, I'm pasting it below for reference
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: pgw-nginx-ingress
namespace: payment-gateway-7682352
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- backend:
serviceName: payment-gateway-dev
servicePort: 5000
path: /dev/
- backend:
serviceName: payment-gateway-qa
servicePort: 5000
path: /qa/
Additional Info
The services I'm trying to access are springboot services that use contexts, so, the root location isn't a valid end-point.
The container's readiness and liveliness probes are defined accordingly.
For example, "payment-gateway-dev" service is using a context /pgw/v1 context, so, the deployment can only be accessed through the context. To access application's swagger spec you would use the URL
http://<>/pgw/v1/swagger-ui.html
Behaviour of my deployment
ingress-controller-LB-ip = 35.188.161.171
Accessing ingress controller load balancer "http://35.188.161.171" takes me to default 404 backend
Accessing ingress controller load balancer health "http://35.188.161.171/healthz" returns 200 HTTP response as expected
Trying to access the services using the URLs below returns "404: page not found" error
http://35.188.161.171/dev/pgw/v1/swagger-ui.html
http://35.188.161.171/qa/pgw/v1/swagger-ui.html
Any suggestions about or insights into what I might be doing wrong will be much appreciated.
+1 for this well asked question.
Your setup seemed right to me. In you explanation, I could find that your services would require http://<>/pgw/v1/swagger-ui.html as context. However, in your setup the path submitted to the service will be http://<>/qa/pgw/v1/swagger-ui.html if your route is /qa/.
To remove the prefix, what you would need to do is to add a rewrite rule to your ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: pgw-nginx-ingress
namespace: payment-gateway-7682352
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- http:
paths:
- backend:
serviceName: payment-gateway-dev
servicePort: 5000
path: /dev/(.+)
- backend:
serviceName: payment-gateway-qa
servicePort: 5000
path: /qa/(.+)
After this, you service should receive the correct contexts.
Ref:
Rewrite: https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/rewrite/README.md
Ingress Route Matching: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/ingress-path-matching.md
An alternate approach would be to use host based routing. You can simply create a couple CNAME DNS entries for the Ingress static IP (pgw-dev.foo.com, pgw-qa.foo.com) and replace your path: attributes with host: attributes. No URL rewrites necessary.
The best reason for using the host based approach, imo, is clarity and flexibility for humans. I've worked in a lot of different places. Almost all of them have used host names to differentiate environments in this way. Tastes great, less filling.
For example, if you split DEV and QA onto separate clusters, no one has to change their configs (and your K8s templates will be reusable). Just update DNS. If you want to spin up a new Staging or Performance Test environment, again, your existing test harnesses should be very easily adapted to the new environment: just change the host name in the config.
Over time, I think you'll find hostname is a more natural way to distinguish environments than a path prefix.