We have a Kubernetes cluster with an nginx controller.
We are using this nginx controller to route different paths to different services. Though,
when a service is on a different namespace, I cannot figure out how to redirect to it.
This is the nginx configuration
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gateway-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/tls-acme: 'true'
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "*"
nginx.ingress.kubernetes.io/cors-allow-headers: "*"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/service-upstream: "true"
spec:
rules:
- host: MY_BACKEND.MY_DOMAIN.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: my-backend
port:
number: 3001
- host: MY_FRONTEND.MY_DOMAIN.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: my-frontend
port:
number: 3000
...
The above snippet works perfectly, but it is worth to mention that both services my-backend and my-frontend are running on default namespace.
We have then deployed a Jenkins instance on namespace jenkins. My first attempty was to simply add
- host: MY_JENKINS.MY_DOMAIN.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: jenkins-service
port:
number: 8080
Though when browsing on MY_JENKINS.MY_DOMAIN.com I get a 503 Service Temporary Unavailable. Could it be because it is running on a different namespace? Namely jenkins?
Normally in order to connect on different namespaces I just use the standard service.namespace. Though here when trying I get a standard console error/warning
The Ingress "gateway-ingress" is invalid: spec.rules[16].http.paths[0].backend.service.name: Invalid value: "WHATEVER_VALUE_WRITTEN": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?')
I tried: service.namespace, namespace.service, namespace/service, service/namespace. All the attempts and up on the same error message.
How can I instruct nginx-controller to connect to a service that is in a specific namespace?
You can reach a service from another namespace with: <service>.<namespace>; by bypassing the service you can also reach other pods by their ip address (even in another namespace).
But afaik all ingress controllers (also nginx) per default do not allow multi-namespace ingress out of security reasons.
For nginx you can configure ingress across multiple namespaces with a master/minion approach. You have to add the nginx.org/mergeable-ingress-type-annotation to your Ingress resources.
A master processes configuration on the host and minions provide the ingress resources.
Find more info here: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/cross-namespace-configuration/
And here:
https://github.com/nginxinc/kubernetes-ingress/tree/v2.2.1/examples/mergeable-ingress-types
Related
I have a domain at Cloudflare and some wildcards for subdomains
which both point to the load balancer of an nginx ingress of a Kubernetes cluster (GKE) of the GCP. Now, we have two pods and services running each (echo1 and echo2, which are essentially identical) and when I apply an ingress
kind: Ingress
metadata:
name: echo-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: "echo1.eu3.example.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo1
port:
number: 80
- host: "echo2.example.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo2
port:
number: 80
I can reach echo2 under echo2.example.com, but not echo1.eu3.example.com. My question is how I can make the second one reachable as well.
I can advise you to make some check.
Just set the Proxy status for "echo1.eu3.example.com" as DNS only. Then check the access. If ok - install certificates in kubernetes via cert manager. We faced some times with this issue and resolved by using 3 deep domains. For instance "echo1-eu3.example.com". It seems cloudfront does not like such domains :) Of course if someone write a solution how to work with deep domains in cloudfront - it would be good practice for us :)
Alright, various permutations of this question have been asked and I feel terrible asking; I'm throwing the towel in and was curious if anyone could point me in the right direction (or point out where I'm wrong). I went ahead and tried a number of examples from the docs, but to no avail (see below).
I'm trying to route traffic to the appropriate location under Kubernetes using an Ingress controller.
Server Setup
I have a server, myserver.com and three services running at:
myserver.com/services/
myserver.com/services/service_1/
myserver.com/services/service_2/
Note that I'm not doing anything (purposefully) to myserver.com/.
At each of the three locations, there's a webapp running. For example, myserver.com/services/service_2 needs to load css files at myserver.com/services/service_2/static/css, etc...
Kubernetes Ingress
To manage the networking, I'm using a Kubernetes Ingress controller, which I've defined below. The CORS annotations aren't super relevant, but I've included them to clear up any confusion.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myServices
namespace: myServices
annotations:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-origin: '$http_origin'
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- myserver.com
rules:
- host: myserver.com
http:
paths:
- path: /services
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80
- path: /services/service_1(/|$)
pathType: Prefix
backend:
service:
name: web-service-1
port:
number: 80
- path: /services/service_2(/|$)
pathType: Prefix
backend:
service:
name: web-service-2
port:
number: 80
Targets
I noticed that one helpful thing to do is give some path examples. From the examples below it looks like the paths aren't that complicated. I think this is what I'm after. Note that I'd like each service to be able to resolve its css and image files.
myserver.com/services -> myserver.com/services
myserver.com/services/xxx/xxx -> myserver.com/services/xxx/xxx
myserver.com/services/service_1 -> myserver.com/services/service_1
myserver.com/services/service_1/xxx/xxx -> myserver.com/services/service_1/xxx/xxx
myserver.com/services/service_2/xxx/xxx -> myserver.com/services/service_2/xxx/xxx
Attempts
I know that this issue has to do a lot with the nginx.ingress.kubernetes.io/rewrite-target rule and its interaction with the paths I've defined.
I know that I don't want nginx.ingress.kubernetes.io/rewrite-target: $1 because that gives a 500 when visiting myserver.com/services
I know that I don't want nginx.ingress.kubernetes.io/rewrite-target: $1/$2 because when I visit myserver.com/services/service_1 I actually get part of the content at myserver.com/services rendered on the page.
SO Attempt 1
I also attempted to replicate the accepted solution from this question.
In this attempt I set
nginx.ingress.kubernetes.io/rewrite-target: "/$1" and one of the service paths to
- path: /(services/service_1(?:/|$).*)
When I visit myserver.com/services/service_1/xyz, the HTML from myserver.com/services/service_1 gets rendered.
Concluding Thoughts
Something ain't quite right with the path rewrite and paths rules. Any suggestions?
The problem you reported in your most recent comment is resolved by looking at the rewrite example in the nginx-ingress documentation.
The rewrite-target annotation configures the ingress such that matching paths will be rewritten to that value. Since you've specified a static value of /, anything matching your ingress rules will get rewritten to /, which is exactly the behavior you're seeing.
The solution is to capture the portion of the path we care about, and then use that in the rewrite-target annotation. For example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myservices
annotations:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-origin: '$http_origin'
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
ingressClassName: nginx
rules:
- host: myserver.com
http:
paths:
- path: /services/service_1(/|$)(.*)
pathType: Prefix
backend:
service:
name: webservice-service1
port:
number: 80
- path: /services/service_2(/|$)(.*)
pathType: Prefix
backend:
service:
name: webservice-service2
port:
number: 80
- path: /services(/|$)(.*)
pathType: Prefix
backend:
service:
name: webservice
port:
number: 80
Here, we've modified the match expression so that they look like:
- path: /services/service_1(/|$)(.*)
The second capture group (.*) captures everything after the path
portion that matches literally. We then use that capture group ($2,
because it's the second group) in the rewrite-target annotation:
nginx.ingress.kubernetes.io/rewrite-target: /$2
With this configuration in place, a request to /services/service_2
results in:
This is service2.
But a request to /services/service_2/foo/bar results in:
<html><head><title>404 Not Found</title></head><body>
<h1>Not Found</h1>
The URL you requested (/foo/bar) was not found.
<hr>
</body></html>
And looking at the backend server logs, we see:
10.42.0.32 - - [21/Jan/2022:20:33:23 +0000] "GET / HTTP/1.1" 200 211 "" "curl/7.79.1"
10.42.0.32 - - [21/Jan/2022:20:33:45 +0000] "GET /foo/bar HTTP/1.1" 404 311 "" "curl/7.79.1"
I've updated my example repository to match this configuration.
I am trying to configure TLS termination via Istio HTTPS -> HTTP.
HTTP 80 works fine.
HTTPS 443 works only for / path.
HTTP 200:
curl https://serviceA.example.com
HTTP 404:
curl https://serviceA.example.com/blabla
Istio access logs:
GET /blabla HTTP/2" 404 NR route_not_found
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: serviceA-gateway
namespace: default
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: HTTP
protocol: HTTP
hosts:
- "serviceA.example.com"
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: serviceA.example.com
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: serviceA-swearl
namespace: default
spec:
hosts:
- serviceA.example.com
gateways:
- serviceA-gateway
HTTP:
- route:
- destination:
host: serviceA.default.svc.cluster.local
port:
number: 80
I am not sure what I did wrong. By looking at the docs everything should be working. Setup is ISTIO operator on AWS EKS with NLB.
Also, I have a certificate - secret in the istio-system namespace.
Service and Deployment have required labels.
FIX: The issue was that I had on Ingress definition
pathType: ImplementationSpecific
It should be:
pathType: Prefix
Configure Ingress pathType ImplementationSpecific behavior #26883
Community wiki answer for better visibility.
As the OP mentioned in the question, problem is solved by setting
pathType: Prefix
in the ingress.
Original message:
FIX: The issue was that I had on Ingress definition
pathType: ImplementationSpecific
It should be pathType: Prefix
https://github.com/istio/istio/issues/26883
You can find an explanation in this official documentation:
Each path in an Ingress is required to have a corresponding path type. Paths that do not include an explicit pathType will fail validation. There are three supported path types:
ImplementationSpecific: With this path type, matching is up to the IngressClass. Implementations can treat this as a separate pathType or treat it identically to Prefix or Exact path types.
Exact: Matches the URL path exactly and with case sensitivity.
Prefix: Matches based on a URL path prefix split by /. Matching is case sensitive and done on a path element by element basis. A path element refers to the list of labels in the path split by the / separator. A request is a match for path p if every p is an element-wise prefix of p of the request path.
I am working with ingress-nginx in kubernetes to set up a server.
The issue is that the paths are not routing at all and I get a 404 error from the nginx server on any request I make.
Below is my code for ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
# defaultBackend:
# service:
# name: auth-srv
# port:
# number: 3000
rules:
- host: app.dev
- http:
paths:
- pathType: Prefix
path: /api/auth/?(.*)
backend:
service:
name: auth-srv
port:
number: 3000
- path: /api/coms/?(.*)
pathType: Prefix
backend:
service:
name: coms-srv
port:
number: 3000
If I uncomment the default backend service I get a response but as soon as I remove it I get the 404 nginx error. So I know its connecting to the services I set.
I don't know where I'm going wrong how to go about fixing this as I'm copying straight from the docs. Any help or insight would be great. Thank you in advance!
Edit 1: I removed the regex from the path and commented out the /api/auth path so no requests should be going to the auth-srv. For some reason, all requests route to the auth-srv even though there is no mapping to it. NOTE: Both the auth and coms pods/services are running in the background, just ingress-nginx still isn't routing properly.
So the reason why this wasn't routing properly was because of the:
- host: app.dev
- http:
The "-" in front of the "http" made the controller think it was its own ruleset so the following routes had a host of "*". After I Removed the "-" in front of the "http", the rules were set to the proper host of app.dev and it started routing accordingly.
Thank you for your help everyone! What a long day it has been :')
I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routingĀ¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.