I'm attempting to connect a client running in a Kubernetes cluster to an Apache Pulsar cluster hosted by StreamNative. Specifically, I'm attempting to use the logstash-input-pulsar Plugin, which doesn't support auth. One option is to fork logstash-input-pulsar and add authentication; however, a more general option would be create a proxy between logstash and pulsar, where the proxy is able to handle authentication. (For example, the proxy could be a sidecar on the kubernetes pod where logstash is running.) I looked into using the Pulsar Proxy; however, this proxy is intended to run on the same Kubernetes cluster as the Pulsar broker(s). If the Pulsar client was using the HTTP protocol, I could set up nginx as a proxy between the client and broker, and nginx could add the appropriate auth - a header, for example. Pulsar, however, uses its own protocol over TCP. Would there still be a way to have a proxy that handles adding auth between the pulsar client and broker?
Related
I have an application that needs to use a proxy (call it proxy1) to access some https endpoints outside of its network. The application doesn't support proxy settings, so I'd like to provide it a reverse proxy url, and I would prefer not to provide tls certs for proxy1, so I would use http for application -> proxy1.
I don't have access to the application host or forward proxy mentioned below, so I cannot configure networking there.
The endpoints the application needs are https, so proxy1 must make its outbound connections via https.
Finally, this whole setup is within a corporate network that requires a forward proxy (call it proxy2) for outbound internet, so my proxy1 needs to chain to proxy2 / use it as a parent.
I tried squid and it worked well for http only, but I couldn't get it to accept http inbound while using https outbound. Squid easily supported the parent proxy2.
I tried haproxy, but had the same result as with squid.
I tried nginx and it did what I wanted with http -> proxy -> https, but doesn't support a parent proxy. I considered setting up socat as in this answer, or using proxy_pass and proxy_set_header as in this answer, but I can't shake the feeling there's a cleaner way to achieve the requirements.
This doesn't seem like an outlandish setup, is it? Or is there a preferred approach for it? Ideally one using squid or nginx.
You can achive this without the complexity by using a port forwarder like socat. Just install it on a host to do the forwarding (or locally on the app server if you wish to) and create a listener that forwards connections through the proxy server. Then on your application host use a local name resolution overide to map the FQDN to the forwarder.
So, the final config should be the app server using a URI that points to the forwarding server (using its address if no name resolution excists), which has a socat listener that points to the the corporate proxy. No reverse proxy required.
socat TCP4-LISTEN:443,reuseaddr,fork \
PROXY:{proxy_address}:{endpoint_fqdn}:443,proxyport={proxy_port}
Just update with your parameters.
I have configured NGINX as a reverse proxy with web sockets enabled for a backend web application with multiple replicas. The request from NGINX does a proxy_pass to a Kubernetes service which in turn load balances the request to the endpoints mapped to the service. I need to ensure that the request from a particular client is proxied to the same Kubernetes back end pod for the life cycle of that access, basically maintaining session persistence.
Tried setting the sessionAffinity: ClientIP in the Kubernetes service, however this does the routing based on the client IP which is of the NGINX proxy. Is there a way to make the Kubernetes service do the affinity based on the actual client IP from where the request originated and not the NGINX internal pod IP ?
This is not an option with Nginx. Or rather it's not an option with anything in userspace like this without a lot of very fancy network manipulation. You'll need to find another option, usually an app-specific proxy rules in the outermost HTTP proxy layer.
Background
I have an in-cluster Kubernetes pod/application that works fine when accessing it via an nginx-ingress ingress controller (requires specific Host HTTP header), but it cannot be accessed by other in-cluster pods/applications (i.e. for testing) due to the pods using different host names (e.g. service-name.namespace.svc.cluster.local) rather than the FQDN of the K8S master (in the LAN).
Plan So Far
I think the only way to (easily) resolve this is to setup an in-cluster forward-proxy nginx instance. Ideally, the service is either a side-car for the pod that needs to have headers re-written, or it needs to be a general in-cluster proxy that multiple services can access.
Question
How would I setup an in-cluster nginx forward proxy service?
Should it be a sidecar, or a general service any pod can access?
Work So Far
The linked "similar" questions don't appear to be helpful for my use case (i.e. don't show how to configure an in-cluster proxy), or are focused on proxying to IPs external to the cluster (i.e. I need to proxy HTTP requests, and re-write their headers, to in-cluster resources).
I wanted to visit my dashboard on a local Kubernetes installation (using docker for mac). I was 'blocked'. I have to provide a token or my config which is normal since the RBAC updates.
Now I don't want to kubectl proxy or enable port forwarding every time I want to visit my dashboard so I installed an nginx proxy with a ingress (tls) which redirects me to https://kubernetes-dashboard.kube-system.svc.cluster.local:443.
This works fine but now I'm a bit confused because I can see the dashboard now, without facing the RBAC issue.
I read this here:
To make Dashboard use authorization header you simply need to pass
Authorization: Bearer in every request to Dashboard. This can
be achieved i.e. by configuring reverse proxy in front of Dashboard.
Proxy will be responsible for authentication with identity provider
and will pass generated token in request header to Dashboard. Note
that Kubernetes API server needs to be configured properly to accept
these tokens.
But it's still not very clear for me. Can someone explain we why I can see the dashboard when I create a proxy in front of it?
Proxy is usually needed to transfer data between different segments of the network without connecting them directly.
Each segment of the network is "talking" to proxy host without any knowledge of the existence of the other network segment.
The Proxy Server is responsible for all negotiations and operations concerning requests and response packets. So, to enable authentication, authorization, SSL termination and many other things you need to configure your proxy server according to your needs.
If you can see the kubernetes dashboard via proxy in front of it it just means that you did not configure any security on that proxy.
For example, to learn how to configure Nginx Ingress to protect a service with basic authentication in your cluster consider to read this article.
For more complex security setup read the article about securing Kubernetes services with Ingress, TLS and LetsEncrypt.
I am not sure how to formulate my question but here we go:
I have 2 servers, one is the nginx reverse proxy and one is the app server.
In my app server, I am developing a simple http client using jerseyclient that will send a request to another server. I can do this now but the traffic goes from the app server and directly to the destination. Is it possible to it from the app server, passes through the reverse proxy server and goes to the destination?
And, is this design ok or is it an abomination?
nginx reverse proxy works only for requests outside your network.
To configure your system works as you described you have to configure firewall NAT or caching HTTP proxy like squid etc.
If you have no reasons why your servers should look as single computer - your configuration is OK.