I am attempting to create an NGINX-INGRESS (locally at first, then to be deployed to AWS behind a load-balancer). However I am new to Kubernetes, and I understand the Ingress model for NGINX- the configurations are confusing me as to weather I should be deploying an NGINX-INGRESS Service, Ingress or Both
I am working with multiple Flask-Apps I would like to have routed by path (/users, /content, etc.) My services are named user-service on port: 8000 (their container port is 8000 as well)
In this example an Ingress is defined. However, when I apply an ingress (in the same Namespace as my Flask there is no response from http://localhost
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-name
namespace: my-namespace
spec:
rules:
- http:
paths:
- path: /users
backend:
serviceName: users-service
servicePort: 8000
- path: /content
backend:
serviceName: content-service
servicePort: 8000
Furthermore, looking at the nginx-ingress "Deployment" docs, under Docker for Mac (which I assume I can use as I am using Docker on a MacOS) they define a Service like so:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
externalTrafficPolicy: Local
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
---
This seems to function for me (When I open "localhost" I get Nginx "not found", but it is a service in a different namespace then my apps- and there is no association between the port 80/443 and my service-ports.
For reference here is one of my deployment/service definitions:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: users-service
labels:
app: users-service
namespace: example
spec:
replicas: 1
selector:
matchLabels:
app: users-service
template:
metadata:
labels:
app: users-service
spec:
containers:
- name: users-service
image: users-service:latest
imagePullPolicy: Never
ports:
- containerPort: 8000
---
kind: Service
apiVersion: v1
metadata:
name: users-service
spec:
selector:
app: users-service
ports:
- protocol: TCP
port: 8000
Update
I followed a video for setting up an NGINX-Controller+Ingress, here the results, entering "localhost/users" does not work,
describe-ingress:
(base) MacBook-Pro-2018-i9:microservices jordanbaucke$ kubectl describe ingress users-ingress
Name: users-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/users users-service:8000 (10.1.0.75:8000)
Annotations: Events: <none>
users-service:
(base) MacBook-Pro-2018-i9:microservices jordanbaucke$ kubectl describe svc users-service
Name: users-service
Namespace: default
Labels: <none>
Annotations: Selector: app=users-service
Type: ClusterIP
IP: 10.100.213.229
Port: <unset> 8000/TCP
TargetPort: 8000/TCP
Endpoints: 10.1.0.75:8000
Session Affinity: None
Events: <none>
nginx-ingress
(base) MacBook-Pro-2018-i9:microservices jordanbaucke$ kubectl describe svc nginx-ingress
Name: nginx-ingress
Namespace: default
Labels: <none>
Annotations: Selector: name=nginx-ingress
Type: NodePort
IP: 10.106.167.181
LoadBalancer Ingress: localhost
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 32710/TCP
Endpoints: 10.1.0.74:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 32240/TCP
Endpoints: 10.1.0.74:443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
Now when I try to enter the combination of NodeIP:NodePort/users, it does not connect?
From inside my nginx-ingress pod, calling:
curl 10.1.0.75:8000 or curl 10.100.213.229:8000 returns results.
For nginx or any other ingress to work properly:
Nginx ingress controller need to deployed on the cluster
A LoadBalancer or NodePort type service need to be created to expose nginx ingress controller via port 80 and 443 in the same namespace where nginx ingress controller is deployed.LoadBalancer works in supported public cloud(AWS etc). NodePort works if running locally.
ClusterIP type service need to be created for workload pods in the namespace where workload pods are deployed.
Workload Pods will be exposed via nginx ingress and you need to create ingress resource in the same namespace as of the clusterIP service of your workload Pods.
You will use either the LoadBalancer(in case nginx ingress controller was exposed via LoadBalancer) or NodeIP:NodePort(in case Nginx ingress controller was exposed via NodePort) to access your workload Pods.
So in this case since docker desktop is being used Loadbalancer type service(ingress-nginx) to expose nginx ingress controller will not work. This needs to be of NodePort type. Once done workload pods can be accessed via NodeIP:NodePort/users and NodeIP:NodePort/content. NodeIP:NodePort should give nginx homepage as well.
Related
I am trying to develop an application on kubernetes with hot-reloading (instant code sync). I am using DevSpace. When running my application on a minikube cluster, everything works and I am able to hit the ingress to reach my FastAPI docs. The problem is when I try to use devspace, I can exec into my pods and see my changes reflected right away, but then when I try to hit my ingress to reach my FastAPI docs, I get a 502 bad gateway.
I have an api-pod.yaml file as such:
apiVersion: apps/v1
kind: Deployment
metadata:
name: project-api
spec:
replicas: 1
selector:
matchLabels:
app: project-api
template:
metadata:
labels:
app: project-api
spec:
containers:
- image: project/project-api:0.0.1
name: project-api
command: ["uvicorn"]
args: ["endpoint:app", "--port=8000", "--host", "0.0.0.0"]
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /api/v1/project/tasks/
port: 8000
initialDelaySeconds: 5
timeoutSeconds: 1
periodSeconds: 600
failureThreshold: 3
ports:
- containerPort: 8000
name: http
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: project-api
spec:
selector:
app: project-api
ports:
- port: 8000
protocol: TCP
targetPort: http
type: ClusterIP
I have an api-ingress.yaml file as such:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: project-ingress
spec:
rules:
- http:
paths:
- path: /api/v1/project/tasks/
pathType: Prefix
backend:
service:
name: project-api
port:
number: 8000
ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: nginx
spec:
controller: k8s.io/ingress-nginx
Using kubectl get ep, I get:
NAME ENDPOINTS AGE
project-api 172.17.0.6:8000 17m
Using kubectl get svc, I get:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
project-api ClusterIP 10.97.182.167 <none> 8000/TCP 17m
Using kubectl get ingress I get:
NAME CLASS HOSTS ADDRESS PORTS AGE
api-ingress nginx * 192.168.64.112 80 17m
to reiterate, my problem is when I try reaching the FastAPI docs using 192.168.64.112/api/v1/project/tasks/docs I get a 502 bad gateway.
Im running:
MacOS Monterey: 12.4
Minikube version: v1.26.0 (with hyperkit as the vm)
Ingress controller: k8s.gcr.io/ingress-nginx/controller:v1.2.1
Devspace version: 5.18.5
I believe the problem was within DevSpace. I am now comfortably using Tilt. Everything is working as expected.
I hope you all are doing great today ,
here is my situation :
i have 2 wordpress websites (identical )
--1st one is an App Service wordpress in azure with a domain name eg:https://wordpress.azurewebsites.net
--the 2nd one is in aks cluster as a pod with a load balancer that expose it to the internet with a public ip
what i want to do :
i want to take the domain name from the app service and give it to the aks pod
what did i do :
i changed from the dashboard the domain name and changed the load balancer public ip adress
and it didn't work now i can't access the dashboard from the load balancer ip adress either
im new in kubernetes i hope someone can guide me to the right direction on how to do it
Seems like you are missing an ingress controller. You could for example install ingress-nginx and expose the ingress with this service config:
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
loadBalancerIP: 53.1.1.1
ports:
- name: https
port: 443
protocol: TCP
targetPort: https
appProtocol: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
You can now create a service for your app:
apiVersion: v1
kind: Service
metadata:
name: app_service
namespace: app
spec:
type: ClusterIP
ports:
- name: service
port: 80
selector:
app: yoour_app
Then you can expose yoour app with an ingress resource:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app_ingress
namespace: app
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- wordpress.azurewebsites.net
rules:
- host: wordpress.azurewebsites.net
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app_service
port:
number: 80
I installed kubernete in several raspberry pi 4 without using cloud.
I follow tutorial to deploy mongodb and mongo express and I can
get inside the mongo express website using node port mode.
However, when I change to use nignx controller, it works when I
type ip address:port (192.168.1.194:30574 It is my master node ip and the port is where the ingress-nginx-controller expose) or domain:port (mymongo.com:30574).
But if I just use domain name (mymongo.com) without port, it cannot get inside the website. Do I miss something? Thank you!
Here is my yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mongo-express-ingress
spec:
rules:
- host: "mymongo.com"
- http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: mongo-express-service
port:
number: 8081
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongo-express
labels:
app: mongo-express
spec:
replicas: 1
selector:
matchLabels:
app: mongo-express
template:
metadata:
labels:
app: mongo-express
spec:
containers:
- name: mongo-express
image: mongo-express
ports:
- containerPort: 8081
env:
- name: ME_CONFIG_MONGODB_ADMINUSERNAME
valueFrom:
secretKeyRef:
name: mongodb-secret
key: mongo-root-username
- name: ME_CONFIG_MONGODB_ADMINPASSWORD
valueFrom:
secretKeyRef:
name: mongodb-secret
key: mongo-root-password
- name: ME_CONFIG_MONGODB_SERVER
valueFrom:
configMapKeyRef:
name: mongodb-configmap
key: database_url
---
apiVersion: v1
kind: Service
metadata:
name: mongo-express-service
spec:
selector:
app: mongo-express
ports:
- protocol: TCP
port: 8081
targetPort: 8081
ubuntu#master:~/k8s-config$ kubectl get services --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
default mongo-express-service ClusterIP 10.107.95.131 <none> 8081/TCP 6d19h
default mongodb-service ClusterIP 10.101.118.57 <none> 27017/TCP 6d19h
ingress-nginx ingress-nginx-controller NodePort 10.96.94.30 <none> 80:30574/TCP,443:31882/TCP 6d4h
ingress-nginx ingress-nginx-controller-admission ClusterIP 10.104.121.41 <none> 443/TCP 6d4h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 8d
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.99.4.159 <none> 8000/TCP 6d22h
kubernetes-dashboard kubernetes-dashboard NodePort 10.98.92.238 <none> 443:31044/TCP
In your current configuration, it works as expected.
Your Nginx Ingress Controller is configured as NodePort Type
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.
So if you are using NodePort type, you will be able to connect only when you will specify <NodeIP:NodePort>, in your case mymongo.com:30574 (NodeIP: mymongo.com, NodePort: 30574).
If you would like to use only a domain, you would need to consider ServiceType as LoadBalancer.
LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.
To achieve that without using Cloud LoadBalancer you can use MetalLB solution.
Your current configuration is also described in Nginx Ingress Documentation as Over a NodePort Service.
You can also consider using host network solution as alternative for baremaetal solution.
In a setup where there is no external load balancer available but using NodePorts is not an option, one can configure ingress-nginx Pods to use the network of the host they run on instead of a dedicated network namespace. The benefit of this approach is that the NGINX Ingress controller can bind ports 80 and 443 directly to Kubernetes nodes' network interfaces, without the extra network translation imposed by NodePort Services.
template:
spec:
hostNetwork: true
i got a bare metal cluster with a few nodeport deployments of my services (http and https). I would like to access them from a single url like myservices.local with (sub)paths.
config could be sth like the following (pseudo code):
/app1
http://10.100.22.55:30322
http://10.100.22.56:30322
# browser access: myservices.local/app1
/app2
https://10.100.22.55:31432
https://10.100.22.56:31432
# browser access: myservices.local/app2
/...
I tried a few things with haproxy and nginx but nothing really worked (for inexperienced in this webserver/lb things kinda confusing syntax/ config style in my opinion).
what is the easiest solution for a case like this?
The easiest and most used way is to use a NGINX Ingress. The NGINX Ingress is built around the Kubernetes Ingress resource, using a ConfigMap to store the NGINX configuration.
In the documentation we can read:
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
internet
|
[ Ingress ]
--|-----|--
[ Services ]
An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.
This is exactly what you want to achieve.
The first thing you need to do is to install the NGINX Ingress Controller in your cluster. You can follow the official Installation Guide.
A ingress is always going to point to a Service. So you need to have a Deployment, a Service and a NGINX Ingress.
Here is an example of an application similar to your example.
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: app1
name: app1
spec:
replicas: 1
selector:
matchLabels:
app: app1
strategy:
type: Recreate
template:
metadata:
labels:
app: app1
spec:
containers:
- name: app1
image: nginx
imagePullPolicy: Always
ports:
- containerPort: 5000
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: app2
name: app2
spec:
replicas: 1
selector:
matchLabels:
app: app2
strategy:
type: Recreate
template:
metadata:
labels:
app: app2
spec:
containers:
- name: app2
image: nginx
imagePullPolicy: Always
ports:
- containerPort: 5001
---
apiVersion: v1
kind: Service
metadata:
name: app1
labels:
app: app1
spec:
type: ClusterIP
ports:
- port: 5000
protocol: TCP
targetPort: 5000
selector:
app: app1
---
apiVersion: v1
kind: Service
metadata:
name: app2
labels:
app: app2
spec:
type: ClusterIP
ports:
- port: 5001
protocol: TCP
targetPort: 5001
selector:
app: app2
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress #ingress resource
metadata:
name: myservices
labels:
app: myservices
spec:
rules:
- host: myservices.local #only match connections to myservices.local.
http:
paths:
- path: /app1
backend:
serviceName: app1
servicePort: 5000
- path: /app2
backend:
serviceName: app2
servicePort: 5001
I have 2 nodes on GCP in a kubernetes cluster. I also have a load balancer in GCP as well. this is a regular cluster (not GCK). I am trying to expose my front-end-service to the world. I am trying nginx-ingress type:nodePort as a solution. Where should my loadbalancer be pointing to? is this a good architecture approach?
world --> GCP-LB --> nginx-ingress-resource(GCP k8s cluster) --> services(pods)
to access my site I would have to point LB to worker-node-IP where nginx pod is running. Is this bad practice. I am new in this subject and trying to understand.
Thank you
deployservice:
apiVersion: v1
kind: Service
metadata:
name: mycha-service
labels:
run: mycha-app
spec:
ports:
- port: 80
targetPort: 3000
protocol: TCP
selector:
app: mycha-app
nginxservice:
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
labels:
app: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31000
port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
name: nginx-ingress
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
labels:
run: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31000
port: 80
targetPort: 3000
protocol: TCP
selector:
app: nginx-ingress
nginx-resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: mycha-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: mycha-service
servicePort: 80
This configuration is not working.
When you use ingress in-front of your workload pods the service type for workload pods will always be of type clusterIP because you are not exposing pods directly outside the cluster.
But you need to expose the ingress controller outside the cluster either using NodePort type service or using Load Balancer type service and for production its recommended to use Loadbalancer type service.
This is the recommended pattern.
Client -> LoadBalancer -> Ingress Controller -> Kubernetes Pods
Ingress controller avoids usage of kube-proxy and load balancing provided by kube-proxy. You can configure layer 7 load balancing in the ingress itself.
The best practise of exposing application is:
World > LoadBalancer/NodePort (for connecting to the cluster) > Ingress (Mostly to redirect traffic) > Service
If you are using Google Cloud Platform, I would use GKE as it is optimized for containers and configure many things automatically for you.
Regarding your issue, I also couldn't obtain IP address for LB <Pending> state, however you can expose your application using NodePort and VMs IP. I will try few other config to obtain ExternalIP and will edit answer.
Below is one of examples how to expose your app using Kubeadm on GCE.
On GCE, your VM already have ExternalIP. This way you can just use Service with NodePort and Ingress to redirect traffic to proper services.
Deploy Nginx Ingress using Helm 3 as tiller is not required anymore ($ helm install nginx stable/nginx-ingress).
As Default it will deploy service with LoadBalancer type but it won't get externalIP and it will stuck in <Pending> state. You have to change it to NodePort and apply changes.
$ kubectl edit svc nginx-nginx-ingress-controller
Default it will open Vi editor. If you want other you need to specify it
$ KUBE_EDITOR="nano" kubectl edit svc nginx-nginx-ingress-controller
Now you can deploy service, deployment and ingress.
apiVersion: v1
kind: Service
metadata:
name: fs
spec:
selector:
key: app
ports:
- port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: fd
spec:
replicas: 1
selector:
matchLabels:
key: app
template:
metadata:
labels:
key: app
spec:
containers:
- name: hello1
image: gcr.io/google-samples/hello-app:1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mycha-deploy
labels:
app: mycha-app
spec:
replicas: 1
selector:
matchLabels:
app: mycha-app
template:
metadata:
labels:
app: mycha-app
spec:
containers:
- name: mycha-container
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: mycha-service
labels:
app: mycha-app
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: mycha-app
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: my.pod.svc
http:
paths:
- path: /mycha
backend:
serviceName: mycha-service
servicePort: 80
- path: /hello
backend:
serviceName: fs
servicePort: 80
service/fs created
deployment.apps/fd created
deployment.apps/mycha-deploy created
service/mycha-service created
ingress.extensions/two-svc-ingress created
$ kubectl get svc nginx-nginx-ingress-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-nginx-ingress-controller NodePort 10.105.247.148 <none> 80:31143/TCP,443:32224/TCP 97m
Now you should use your VM ExternalIP (slave VM) with port from NodePort service. My VM ExternalIP: 35.228.133.12, service: 80:31143/TCP,443:32224/TCP
IMPORTANT
If you would curl your VM with port you would get response:
$ curl 35.228.235.99:31143
curl: (7) Failed to connect to 35.228.235.99 port 31143: Connection timed out
As you are doing this manually, you also need add Firewall rule to allow traffic from outside on this specific port or range.
Information about creation of Firewall Rules can be found here.
If you will set proper values (open ports, set IP range (0.0.0.0/0), etc) you will be able to get service from you machine.
Curl from my local machine:
$ curl -H "HOST: my.pod.svc" http://35.228.235.99:31143/mycha
<!DOCTYPE html>
...
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
$ curl -H "HOST: my.pod.svc" http://35.228.235.99:31143/hello
Hello, world!
Version: 1.0.0
Hostname: fd-c6d79cdf8-dq2d6