I am trying to configure Kafka cluster behind nginx stream.
My kafka is running in VM inside a VNet (intranet zone), I have another VM in which nginx is runnig (internet Zone). I want to be able to connect kafka through proxy. I am not sure how to configure the advertised listeners.
suppose Nginx server is listening on port 8082, and my broker is listening in port 9092. Kindly help me .
Thanks
Related
Here is what i have:
GCP instance without external IP (on VPC, and NAT), and it accepts HTTP HTTPS requests
firewall allows ingress TCP for 0.0.0.0 and also for IAP's IP 35.235.240.0/20 on all ports for all instances
I ssh to the instance via IAP and run the application in the terminal on port 5000 and 0.0.0.0 host and leave the terminal hanging, but when I connect in parallel through cloud shell and ssh to this instance through IAP, and then click on web preview on port 5000, I get "Couldn't connect to a server on port 5000".
I have said that it could be a firewall rule blocking IAP, so that's why I gave access to all ports for IAP (for testing)
P.S: the process has been done on a VM with external IP and it got validated ( but without the need to connect to cloud shell to do web preview, I checked the UI with IP:port in the browser )
What did I miss?
You may be following the guide on Building Internet Connectivity for private VMs and this part on Configuring IAP tunnels for interacting with instances and the use of TCP Forwarding in IAP. By Tunneling other TCP connections:
"The local port tunnels data traffic from the local machine to the remote machine in an HTTPS stream. IAP then receives the data, applies access controls, and forwards the unwrapped data to the remote port."
You can create an encrypted tunnel to a port of the VM instance by:
gcloud compute start-iap-tunnel INSTANCE_NAME INSTANCE_PORT \
--local-host-port=localhost:LOCAL_PORT \
--zone=ZONE
I guess you want to use INSTACE_PORT and LOCAL_PORT the same, 5000.
Be aware of it's known limitations.
I have created a local websocket server using the ktor framework, which is available locally in the below IP.
[main] INFO Application - Responding at http://192.168.1.10:8080
And I'm able to access it locally from postman by hitting
ws://192.168.1.10:8080/chat
I would like to connect to this socket from the internet, But when I setup port forwarding (using NAT virtual server configuration), this doesn't seem to work. I get timed out error.
I tried this because, it is working fine for the http server (express app) I created in another port. From the below NAT virtual server configuration, I was able to do port forwarding and access the webservice from the internet.
I'm quite new to socket programming and networking. Kindly advice.
You can't have a single External Port mapped to two different Server ports. Each Server needs a distinct External Port mapped to it. Right now, you have External Port 80 mapped to both 192.168.1.10:3000 and 192.168.1.10:8080.
Once you fix that issue to use a separate External Port for each Server, then http://<externalIP>:<externalPort1> and ws://<externalIP>:<externalPort2> should work properly to reach the HTTP and WebSocket servers, respectively, from the outside world.
I am trying to run an test app on pivotal cloud foundry, that listens on TCP port 2222. I was able to create a TCP route however i was not able to reach this server from a client outside, any ideas? on the same vm i am able to connect.
You need to talk to your platform operator. In addition to mapping a TCP route to your app, you need to have load balancers configured to route TCP traffic into the platform.
Please check with your platform operator to see if the following steps have been performed.
https://docs.pivotal.io/pivotalcf/2-3/adminguide/enabling-tcp-routing.html
I setup an Elastic Beanstalk with load balancer forwarding port 80 to port 5000 on EC2 instance. My EC2 instance listens on port 5000, not port 80. The EC2 instance has a private ip 172.31.14.151. On another EC2 which is in the same subnet as the EC2 running the Springboot web server, I got http responses for the two following http request:
curl 172.31.14.151:5000
curl 172.31.14.151:80
I do not understand why I got http response from 172.31.14.15:80. The EC2 I am running the curl command is on the same subnet as the EC2 running webserver. The http request should not go through any router and not through load balancer. But the webserver is running on port 5000, not port 80.
Is there a Nginx instance running on the EC2 instance with webserver?
If I configure the webserver to listen on port 80 and let the Elastic loadbalancer forward port 80 to port 80 on EC2 instance, I got Nginx 502 bad gateway response from doing the curl request
curl 172.31.14.151:80
I don't know which Elastic Beanstalk Solution Stack you are using, but most of the AWS Solution Stacks come coupled with Proxy Servers by default. For example, if you're running Java SE the proxy server is NGINX, but if you're running Java with Tomcat the proxy server is Apache.
By default these proxies accept HTTP Traffic on the default HTTP port (80), manage the connections, then proxy the requests from the backing application server (In your case, port 5000). This helps manage the connection to the backing application, as well as serve static content, or if you configure them correctly, customized Error messages based on the HTTP Status code. I'd suggest that if you can, you send the load balancer traffic to port 80 because Apache or NGINX can usually handle connection load better than most custom applications.
Have you check inbound rules on the security group that you've use ?
Is there a Nginx instance running on the EC2 instance with webserver? - Yes is it. When you create new environment, you can choose pre-configured platform, and choose NodeJS Platform.
If your application is heterogeneous applications, is better to use container. You can deploy your containerized applications on Elasticbeanstalk or use Elastic Container Service instead.
Suppose I want to create a k8s cluster on bare metal servers, with 1 master and 2 nodes. What ports do I have to open in my firewall so that the master and nodes can communicate over the Internet? (I know I can just use VPN, but I just want to know which ports I need). I guess I need at least the following ports. Do I need more? How about if I'm using Flannel or Calico? I want to create a comprehensive list of all possible k8s services and needed ports. Thank you.
kubectl - 8080
ui - 80 or 443 or 9090
etcd - 2379, 2380
the ports for kubernetes are the following:
from the CoreOS docs.
Kubernestes needs:
Master node(s):
TCP 6443* Kubernetes API Server
TCP 2379-2380 etcd server client API
TCP 10250 Kubelet API
TCP 10251 kube-scheduler
TCP 10252 kube-controller-manager
TCP 10255 Read-Only Kubelet API
Worker nodes (minions):
TCP 10250 Kubelet API
TCP 10255 Read-Only Kubelet API
TCP 30000-32767 NodePort Services
Providing that the API server, etcd, scheduler and controller manager run on the same machine, the ports you would need to open publicly in the absence of VPN are:
Master
6443 (or 8080 if TLS is disabled)
Client connections to the API server from nodes (kubelet, kube-proxy, pods) and users (kubectl, ...)
Nodes
10250 (insecure by default!)
Kubelet port, accepts connections from the API server (master).
Also nodes should be able to receive traffic from other nodes and from the master on pretty much any port, on the network fabric used for Kubernetes pods (flannel, weave, calico, ...)
If you expose applications using a NodePort service or Ingress resource, the corresponding ports should also be open on your nodes.