I am having trouble egetting through with gRPC from external to istio-ingress on Kubernetes.
But I have come so fare, that I can get a respons 200. Now I expect that HTTP/1.1 is curl not supporting http2.
Any help is appreciated, thanks!
HTTP/1.1 200 OK
content-type: application/grpc
grpc-status: 8
grpc-message: malformed method name: "/ghw"
x-envoy-upstream-service-time: 8
date: Thu, 03 May 2018 18:33:28 GMT
server: envoy
content-length: 0
The yaml setup is as following:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: grpc-deployment
labels:
app: grpc
spec:
selector:
matchLabels:
app: grpc
replicas: 1
template:
metadata:
labels:
app: grpc
spec:
containers:
- name: grpc
image: local/gcd
imagePullPolicy: Never
ports:
- name: grpc-port
containerPort: 3000
# protocol: HTTP2
---
apiVersion: v1
kind: Service
metadata:
name: grpc-service
spec:
# type: LoadBalancer
selector:
app: grpc
ports:
- port: 3000
name: grpc
# protocol: HTTP2
targetPort: 3000
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grpc-ingress
annotations:
kubernetes.io/ingress.class: "istio"
# ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- http:
paths:
- path: /ghw
backend:
serviceName: grpc-service
servicePort: 3000
Related
Hopefully a simple one to answer.
I've deployed a test Nginx Deployment, Service, and Ingress with the agent sidecar annotation, but it's not appearing in Jaeger-Query.
I've followed this section of the docs: https://www.jaegertracing.io/docs/1.37/operator/#auto-injecting-jaeger-agent-sidecars
My Nginx .yaml file is configured as below:
apiVersion: apps/v1
kind: Deployment
metadata:
name: jaeger-nginx-test-deployment
namespace: observability
annotations:
sidecar.jaegertracing.io/inject: "true"
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx:1.14.2
name: jaeger-nginx-test-deployment
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: jaeger-nginx-test
namespace: observability
labels:
app: nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
app: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jaeger-nginx-test-ingress
namespace: observability
annotations:
kubernetes.io/ingress.class: "gce"
spec:
rules:
- http:
paths:
- path: "/*"
pathType: ImplementationSpecific
backend:
service:
name: jaeger-nginx-test
port:
number: 80
Could someone please advise how we can get this to appear in the Jaeger-Query UI?
At the moment it only recognises the 'jaeger-query' service.
The below example worked for me after running:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.1/deploy/static/provider/cloud/deploy.yaml
kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.37.0/jaeger-operator.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: jaeger-test-deployment
namespace: observability
annotations:
sidecar.jaegertracing.io/inject: "true"
spec:
selector:
matchLabels:
app: test-deployment
replicas: 1
template:
metadata:
labels:
app: test-deployment
spec:
containers:
- name: jaeger-test-deployment
image: jaegertracing/example-hotrod:1.28
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: jaeger-test-deployment
namespace: observability
labels:
app: test-deployment
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
app: test-deployment
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jaeger-test-ingress
namespace: observability
annotations:
kubernetes.io/ingress.class: "gce"
spec:
rules:
- http:
paths:
- path: "/*"
pathType: ImplementationSpecific
backend:
service:
name: jaeger-test-deployment
port:
number: 80
I'm trying to run a minimalistic sample of oauth2-proxy with Keycloak. I used oauth2-proxy's k8s example, which uses dex, to build up my keycloak example.
The problem is that I don't seem to get the proxy to work:
# kubectl get pods
NAME READY STATUS RESTARTS AGE
httpbin-774999875d-zbczh 1/1 Running 0 2m49s
keycloak-758d7c758-27pgh 1/1 Running 0 2m49s
oauth2-proxy-5875dd67db-8qwqn 0/1 CrashLoopBackOff 2 2m49s
Logs indicate a network error:
# kubectl logs oauth2-proxy-5875dd67db-8qwqn
[2021/09/22 08:14:56] [main.go:54] Get "http://keycloak.localtest.me/auth/realms/master/.well-known/openid-configuration": dial tcp 127.0.0.1:80: connect: connection refused
I believe I have set up the ingress correctly, though.
Steps to reproduce
Set up the cluster:
#Creare kind cluster
wget https://raw.githubusercontent.com/oauth2-proxy/oauth2-proxy/master/contrib/local-environment/kubernetes/kind-cluster.yaml
kind create cluster --name oauth2-proxy --config kind-cluster.yaml
#Setup dns
wget https://raw.githubusercontent.com/oauth2-proxy/oauth2-proxy/master/contrib/local-environment/kubernetes/custom-dns.yaml
kubectl apply -f custom-dns.yaml
kubectl -n kube-system rollout restart deployment/coredns
kubectl -n kube-system rollout status --timeout 5m deployment/coredns
#Setup ingress
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml
kubectl --namespace ingress-nginx rollout status --timeout 5m deployment/ingress-nginx-controller
#Deploy
#import keycloak master realm
wget https://raw.githubusercontent.com/oauth2-proxy/oauth2-proxy/master/contrib/local-environment/keycloak/master-realm.json
kubectl create configmap keycloak-import-config --from-file=master-realm.json=master-realm.json
Deploy the test application. My deployment.yaml file:
###############oauth2-proxy#############
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
name: oauth2-proxy
name: oauth2-proxy
spec:
replicas: 1
selector:
matchLabels:
name: oauth2-proxy
template:
metadata:
labels:
name: oauth2-proxy
spec:
containers:
- args:
- --provider=oidc
- --oidc-issuer-url=http://keycloak.localtest.me/auth/realms/master
- --upstream="file://dev/null"
- --client-id=oauth2-proxy
- --client-secret=72341b6d-7065-4518-a0e4-50ee15025608
- --cookie-secret=x-1vrrMhC-886ITuz8ySNw==
- --email-domain=*
- --scope=openid profile email users
- --cookie-domain=.localtest.me
- --whitelist-domain=.localtest.me
- --pass-authorization-header=true
- --pass-access-token=true
- --pass-user-headers=true
- --set-authorization-header=true
- --set-xauthrequest=true
- --cookie-refresh=1m
- --cookie-expire=30m
- --http-address=0.0.0.0:4180
image: quay.io/oauth2-proxy/oauth2-proxy:latest
# image: "quay.io/pusher/oauth2_proxy:v5.1.0"
name: oauth2-proxy
ports:
- containerPort: 4180
name: http
protocol: TCP
livenessProbe:
httpGet:
path: /ping
port: http
scheme: HTTP
initialDelaySeconds: 0
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /ping
port: http
scheme: HTTP
initialDelaySeconds: 0
timeoutSeconds: 1
successThreshold: 1
periodSeconds: 10
resources:
{}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: oauth2-proxy
name: oauth2-proxy
spec:
type: ClusterIP
ports:
- port: 4180
targetPort: 4180
name: http
selector:
name: oauth2-proxy
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
app: oauth2-proxy
name: oauth2-proxy
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
large_client_header_buffers 4 32k;
spec:
rules:
- host: oauth2-proxy.localtest.me
http:
paths:
- path: /
backend:
serviceName: oauth2-proxy
servicePort: 4180
---
# ######################httpbin##################
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
selector:
matchLabels:
name: httpbin
template:
metadata:
labels:
name: httpbin
spec:
containers:
- image: kennethreitz/httpbin:latest
name: httpbin
resources: {}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
hostname: httpbin
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: httpbin-svc
labels:
app: httpbin
spec:
type: ClusterIP
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
selector:
name: httpbin
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: httpbin
labels:
name: httpbin
annotations:
nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-User,X-Auth-Request-Email
nginx.ingress.kubernetes.io/auth-signin: http://oauth2-proxy.localtest.me/oauth2/start
nginx.ingress.kubernetes.io/auth-url: http://oauth2-proxy.localtest.me/oauth2/auth
spec:
rules:
- host: httpbin.localtest.me
http:
paths:
- path: /
backend:
serviceName: httpbin-svc
servicePort: 80
---
# ######################keycloak#############
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: keycloak
name: keycloak
spec:
replicas: 1
selector:
matchLabels:
app: keycloak
template:
metadata:
labels:
app: keycloak
spec:
containers:
- args:
- -Dkeycloak.migration.action=import
- -Dkeycloak.migration.provider=singleFile
- -Dkeycloak.migration.file=/etc/keycloak_import/master-realm.json
- -Dkeycloak.migration.strategy=IGNORE_EXISTING
env:
- name: KEYCLOAK_PASSWORD
value: password
- name: KEYCLOAK_USER
value: admin#example.com
- name: KEYCLOAK_HOSTNAME
value: keycloak.localtest.me
- name: PROXY_ADDRESS_FORWARDING
value: "true"
image: quay.io/keycloak/keycloak:15.0.2
# image: jboss/keycloak:10.0.0
name: keycloak
ports:
- name: http
containerPort: 8080
- name: https
containerPort: 8443
readinessProbe:
httpGet:
path: /auth/realms/master
port: 8080
volumeMounts:
- mountPath: /etc/keycloak_import
name: keycloak-config
hostname: keycloak
volumes:
- configMap:
defaultMode: 420
name: keycloak-import-config
name: keycloak-config
---
apiVersion: v1
kind: Service
metadata:
name: keycloak-svc
labels:
app: keycloak
spec:
type: ClusterIP
sessionAffinity: None
ports:
- name: http
targetPort: http
port: 8080
selector:
app: keycloak
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: keycloak
spec:
tls:
- hosts:
- "keycloak.localtest.me"
rules:
- host: "keycloak.localtest.me"
http:
paths:
- path: /
backend:
serviceName: keycloak-svc
servicePort: 8080
---
# kubectl apply -f deployment.yaml
Configure /etc/hosts on the development machine file to include localtest.me domain:
127.0.0.1 oauth2-proxy.localtest.me
127.0.0.1 keycloak.localtest.me
127.0.0.1 httpbin.localtest.me
127.0.0.1 localhost
Note that I can reach http://keycloak.localtest.me/auth/realms/master/.well-known/openid-configuration with no problem from my host browser. It appears that the oauth2-proxy's pod cannot reach the service via the ingress. Would really appreciate any sort of help here.
Turned out that I needed to add keycloak to custom-dns.yaml.
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
hosts {
10.244.0.1 dex.localtest.me. # <----Configured for dex
10.244.0.1 oauth2-proxy.localtest.me
fallthrough
}
}
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
Added keycloak showed as below:
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
hosts {
10.244.0.1 keycloak.localtest.me
10.244.0.1 oauth2-proxy.localtest.me
fallthrough
}
}
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
I have 2 websites. Each of the site has their corresponding subdomain. To achieve this in AKS, I'm using nginx-ingress to route the traffic according to incoming subdomain.
testing-web1 - http://web1.testing.com/
testing-web2 - http://web2.testing.com/
testing-web1 - by IP address (for testing purpose)
When I enter http://web1.testing.com/, nginx-ingress is routing me to testing-web1. If I enter http://web2.testing.com/, nginx-ingress will route me to testing-web1 instead of testing-web2.
Is this an expected behavior? Did I miss-configure something? I think I'm almost there, but I couldn't figure out what went wrong.
Thanks.
apiVersion: apps/v1
kind: Deployment
metadata:
name: testing-web1
spec:
replicas: 1
selector:
matchLabels:
app: testing-web1
minReadySeconds: 5
template:
metadata:
labels:
app: testing-web1
spec:
nodeSelector:
"beta.kubernetes.io/os": linux
containers:
- name: testing-web1
image: image-web1:1.0
ports:
- containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: testing-web2
spec:
replicas: 1
selector:
matchLabels:
app: testing-web2
minReadySeconds: 5
template:
metadata:
labels:
app: testing-web2
spec:
nodeSelector:
"beta.kubernetes.io/os": linux
containers:
- name: testing-web2
image: image-web2:1.0
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: testing-web1
labels:
app: testing-web1
spec:
ports:
- port: 80
selector:
app: testing-web1
---
apiVersion: v1
kind: Service
metadata:
name: testing-web2
labels:
app: testing-web2
spec:
ports:
- port: 80
selector:
app: testing-web2
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp-ingress
annotations:
kubernetes.io/ingress.class: nginx
certmanager.k8s.io/cluster-issuer: letsencrypt-staging
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- web1.testing.com
- web2.testing.com
rules:
- host: web1.testing.com
http:
paths:
- path: /
backend:
serviceName: testing-web1
servicePort: 80
- host: web2.testing.com
http:
paths:
- path: /
backend:
serviceName: testing-web2
servicePort: 80
- http:
paths:
- backend:
serviceName: testing-web1
servicePort: 80
path: /
I want to set up an Ingress, which routes traffic to my underlying Services. Unfortunately, I get an error when I deploy my ingress-controller-deployment.yaml and I don't know why... The pod with the ingress-controller crashes immediately, with the error message "CrashLoopBackOff".
With my understanding the Ingress-Control has to be deployed in a Pod and this pod can be accessed through the ingress-svc. The ingress-svc seems to work, but the Pod crashes. After the ingress-controller works I need an additional file that defines the routes and everything. But I don't see the point of continuing with out a working and deployable ingress-controller.
Pod description:
Name: ingress-controller-7749c785f-x94ll
Namespace: ingress
Node: gke-cluster-1-default-pool-8484e77d-r4wp/10.128.0.2
Start Time: Thu, 26 Apr 2018 14:25:04 +0200
Labels: k8s-app=nginx-ingress-lb
pod-template-hash=330573419
Annotations: kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"ingress","name":"ingress-controller-7749c785f","uid":"d8ff0a6d-494c-11e8-a840
-420...
Status: Running
IP: 10.8.0.14
Created By: ReplicaSet/ingress-controller-7749c785f
Controlled By: ReplicaSet/ingress-controller-7749c785f
Containers:
nginx-ingress-controller:
Container ID: docker://5654c7dffc44510132cba303d66ee570280f2cec235e4d4fa6ef8ad543e0c91d
Image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0
Image ID: docker-pullable://quay.io/kubernetes-ingress-controller/nginx-ingress-controller#sha256:39cc6ce23e5bcdf8aa78bc28bbcfe0999e449bf99fe2e8d60984b417facc5cd4
Ports: 80/TCP, 443/TCP
Args:
/nginx-ingress-controller
--admin-backend-svc=$(POD_NAMESPACE)/admin-backend
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 2
Started: Thu, 26 Apr 2018 14:26:57 +0200
Finished: Thu, 26 Apr 2018 14:26:57 +0200
Ready: False
Restart Count: 4
Liveness: http-get http://:10254/healthz delay=10s timeout=5s period=10s #success=1 #failure=3
Environment:
POD_NAME: ingress-controller-7749c785f-x94ll (v1:metadata.name)
POD_NAMESPACE: ingress (v1:metadata.namespace)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-plbss (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
default-token-plbss:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-plbss
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.alpha.kubernetes.io/notReady:NoExecute for 300s
node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Ingress-controller-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ingress-controller
spec:
replicas: 1
revisionHistoryLimit: 3
template:
metadata:
labels:
k8s-app: nginx-ingress-lb
spec:
containers:
- args:
- /nginx-ingress-controller
- "--admin-backend-svc=$(POD_NAMESPACE)/admin-backend"
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0"
imagePullPolicy: Always
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
timeoutSeconds: 5
name: nginx-ingress-controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: ingress-svc
spec:
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
selector:
k8s-app: nginx-ingress-lb
The issue is the args. The args on one of mine are
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
I had also created the config maps for configuration, tcp and udp.
From Kubernetes v1.6, RBAC authorize feature is enabled by default. This implies that the deployments/configurations I had for v1.5, are no longer working.
One of the key components to which I needed to grant access is to nginx, otherwise a message like to following can be seen on the logs
F0425 15:08:07.246596 1 main.go:116] no service with name kube-system/default-http-backend found: the server does not allow access to the requested resource (get services default-http-backend)
UPDATED: kubernetes/nginx has the documentation updated here and for RBAC details, here
OLD:
In order to support RBAC, we need two things:
define the servciceAccount/ClusterRole/ClusterRoleBindings
set a serviceAccount for the nginx deployment
Here are the files I use to set it up:
nginx-roles.yml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: nginx-role
rules:
- apiGroups: [""]
resources: ["secrets", "configmaps", "services", "endpoints"]
verbs:
- get
- watch
- list
- proxy
- use
- redirect
- apiGroups: [""]
resources: ["events"]
verbs:
- redirect
- patch
- post
- apiGroups:
- "extensions"
resources:
- "ingresses"
verbs:
- get
- watch
- list
- proxy
- use
- redirect
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: nginx-role
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-role
subjects:
- kind: ServiceAccount
name: nginx
namespace: kube-system
nginx-ingress-controller.yml
with nodeSelector: kubecluster-amd-1 and default-http-backend used
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
labels:
k8s-app: nginx-ingress-controller
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: nginx-ingress-controller
spec:
serviceAccount: nginx
hostNetwork: true
nodeSelector:
kubernetes.io/hostname: kubecluster-amd-1
terminationGracePeriodSeconds: 60
containers:
- image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.4
name: nginx-ingress-controller
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 20
timeoutSeconds: 1
ports:
- containerPort: 80
hostPort: 80
- containerPort: 443
hostPort: 443
- containerPort: 5683
hostPort: 5683
protocol: UDP
- containerPort: 5684
hostPort: 5684
protocol: UDP
- containerPort: 53
hostPort: 53
protocol: UDP
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend