In my environment, elasticsearch sits on a server that only has standard ports (80, 443, etc.) open. All the other ports are firewalled off. I currently have a reverse proxy on port 80 that reroutes all the elasticsearch HTTP requests to elasticsearch's http port.
I would also like to reroute TCP requests to elasticsearch's transport port, so that my local client can directly query elasticsearch as a client node. Nginx 1.9.0 recently allowed TCP load balancing, which is what I would like to utilize for this, but I'm having some trouble getting my system to work. Here is my nginx.conf file (removed the HTTP context to isolate the issue):
worker_processes 1;
events {
worker_connections 1024;
}
stream {
server {
listen 80;
proxy_pass 127.0.0.1:9300;
}
}
My client node is set up to talk to mydomain.com:80, so it should ideally be routing all traffic to the internal transport port. However, I am getting a the following exceptions: org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available
Is there something else I need to configure on my client node or the tcp proxy?
EDIT 1:
Some additional information. I changed Elasticsearch's transport port from 9300 to 8030, which is a port that is open. When I correspondingly changed my nginx.conf to proxypass to 127.0.0.1:8030 my local client node started working, and got appropriate responses to my queries.
So the issue seems to be that if I'm proxy pass to an already open port, it works, but if the port is closed (9300), the proxy pass fails. Does anyone know why this would be and how to fix it? I'd prefer to stick to using port 9300 if possible.
Related
Good day,
I want to connect to my kafka server from the internet. Kafka installed on the virtual server and all servers hidden behind a nginx.
I updated kafka settings (server.properties).
Added: listeners=PLAINTEXT://:9092
I can connect to kafka server from local network via ip address 10.0.0.1:9092, but unable connect from internet by domain name.
Response from kafka: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic test-topic not present in metadata after 60000 ms.
Nginx: [26/Nov/2019:12:38:25 +0100] "\x00\x00\x00\x14\x00\x12\x00\x02\x00\x00\x00\x00\x00" 400 166 "-" "-" "request_time=1.535" "upstream_response_time=-" "upstream_connect_time=-" "upstream_header_time=-"
nginx conf:
server {
listen 9092;
server_name site.name;
# Max Request size
client_max_body_size 20m;
location / {
proxy_pass http://10.0.0.1:9092;
}
}
Does anyone know what the problem is?
Kafka doesn't use http protocol for communication, so it can't be fronted by an HTTP reverse proxy.
You'll have to use nginx stream definition blocks for TCP proxying
(I've not tried this personally)
https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
unable connect from internet by domain name.
Sounds like an issue with your advertised.listeners configuration. Note that there is no clear way to "hide" Kafka behind a proxy since your clients are required to communicate directly with each broker individually (therefore defeating the purpose of having
Ngnix unless you want to use one Nginx server or open a new port, per broker), and would therefore also require Kafka to know that it would need to "advertise" the proxy rather than its own address.
If you really want to expose Kafka to the public web, you should really be using SSL/SASL listeners, not PLAINTEXT
If you want to use HTTP, then you can install Kafka REST Proxy, then put Nginx in front of that. Then your clients would use http rather than standard kafka libraries
I have 2 logstash servers that listen on 5000 from multiple filebeats. Only one of the logstash servers is getting traffic. Can I loadbalance these with nginx? Can I do the same with a cluster of elasticsearch servers?
That doesn't sound like an issue.
I might suggest using haproxy as the reverse proxy in both cases though. Nginx has a good rep for serving data fast, but not as good with reverse proxy out of the box - lots of 504 gateway timeout issues and minimal metrics compared to haproxy.
You could configure the same proxy to be used for the logstash and elasticsearch server group.
I got it to work. I had a version of nginx that was built without the --with-stream tags that allow me to proxy tcp traffic. works now.
There should be no problem doing load-balancing using NginX for the Logstash. Just be aware that you need to use stream block instead of http to do load-balancing in tcp/udp layer. That's because filebeat uses a protocol called lumberjack to connect logstash server. This protocol sits on the top of tcp. So, your NginX configuration should look like this:
stream {
upstream logstash {
server logstash-1:5044;
server logstash-2:5044;
}
server {
listen 5044;
listen [::]:5044;
proxy_pass logstash;
}
}
Here I used docker to run logstash-1 and logstash-2 on separate containers so they can be on the same 5044 port. Feel free to change these addresses based on your specific logstash hosts and ports.
I'm trying to setup an FTP subdomain, such that all incoming SFTP requests to (say) ftp.myname.com, get routed to a particular internal server, (say) 10.123.456 via port 22.
How do I use nginx to route this traffic?
I've already setup the SFTP server, and can SFTP directly to the server, say:
sftp username#123.456.7890, which works fine.
The problem is that when I setup nginx to route all traffic to ftp.myname.com, it connects, but the passwords get rejected. I have no problems routing web traffic to my other subdomains, say dev.myname.com (with passwords), but it doesn't work for the SFTP traffic:
server {
listen 22;
server_name ftp.myname.com;
return .............
}
How do I define the return sting to route the traffic with the passwords?
The connection is SFTP (via port 22).
Thanks
Aswering to #peixotorms: yes, you can. nginx can proxy/load balance http as well as tcp and udp traffic, see nginx stream modules documentation (at the nginx main documentation page) , and specifically the stream core module's documentation.
You cannot do this on nginx (http only), you must use something like HaProxy and a simple dns record for your subdomain pointing to the server ip.
Some info: http://jpmorris-iso.blogspot.pt/2013/01/load-balancing-openssh-sftp-with-haproxy.html
Edit:
Since nginx version 1.15.2 it's now possible to do that using the variable $ssl_preread_protocol. The official blog added post about how to use this variable for multiplexing HTTPS and SSH on the same port.
https://www.nginx.com/blog/running-non-ssl-protocols-over-ssl-port-nginx-1-15-2/
Example of configuring SSH on an upstream block:
stream {
upstream ssh {
server 192.0.2.1:22;
}
upstream sslweb {
server 192.0.2.2:443;
}
map $ssl_preread_protocol $upstream {
default ssh;
"TLSv1.2" sslweb;
}
# SSH and SSL on the same port
server {
listen 443;
proxy_pass $upstream;
ssl_preread on;
}
}
Theres a server in a customer that runs a nginx, a salt master daemon from saltstack and a secret web app that does secret things.
Considerations:
In this scenario, theres only one ip, only one server and multiple DNS records available;
I have nginx running in port 80;
And salt master running in 6453;
A domain.example.com binding to that IP, exposing my nginx 80 port, that points to the secret webapp;
otherdomain.example.com binding to the same IP, exposing my nginx 80 port, that I want to use to proxy salt port.
That customer has a machine in other place, that does need to connect to the salt and the internet connection is provided by a secret organization and they only allow connections to port 80, no negotiation possible.
My question:
Is possible to use nginx to redirect the otherdomain.example.com 80 port to the 6453 port? I tried the following:
server {
listen 80;
server_name otherdomain.example.com;
proxy_pass 127.0.0.1:6453;
}
But that doesn't work as expected. It is possible? There's some way to do this using nginx?
The error I got from log was:
"proxy_pass" directive is not allowed here
proxy_pass needs to be specified within a location context, and is fundamentally a Web Thing. It only comes into play after the web headers are sent and interpreted.
Things like what you're trying to accomplish are commonly done using HAProxy in tcp mode, although there is a tcp proxy module that also does similar things.
However, I don't think you're going to be very successful, as ZMQ does not participate in the protocol (HTTP Host: headers) that easily allows you to tell the web requests apart from the non-web requests (that come in on the same port).
My recommendation is to either find some way to use another port for this, a second IP address, or write a tricky TCP proxier that'll identify incoming HTTP and/or ZMQ connections and transparently forward them to the correct local port.
I'm trying to run two Minecraft servers on the same machine on two different ports. I want to reference them based on subdomains:
one.example.com -> <minecraft>:25500
two.example.com -> <minecraft>:25501
I have used nginx for things like this before, but it's not working with Minecraft. It's responding with http status 400. Here is a sample from my log:
192.168.0.1 - - [21/Apr/2013:17:25:40 -0700] "\x02<\x00\x0E\x00t\x00h\x00e\x00s\x00a\x00n\x00d\x00y\x00m\x00a\x00n\x001\x002\x003\x00\x1C\x00t\x00e\x00s\x00t\x00.\x00r\x00y\x00a\x00n\x00s\x00a\x00n\x00d\x00y\x00.\x00i\x00s\x00-\x00a\x00-\x00g\x00e\x00e\x00k\x00.\x00c\x00o\x00m\x00\x00c\xDD" 400 173 "-" "-"
Here is my nginx config:
upstream mine1 {
server 127.0.0.1:25500;
}
upstream mine2 {
server 127.0.0.1:25501;
}
server {
listen 25565;
server_name one.example.com;
access_log /var/log/nginx/one.access;
error_log /var/log/nginx/one.error;
location / {
proxy_pass http://mine1;
}
}
server {
listen 25565;
server_name two.example.com;
access_log /var/log/nginx/two.access;
error_log /var/log/nginx/two.error;
location / {
proxy_pass http://mine2;
}
}
If I'm reading this correctly, nginx is responding with 400. My guess is the Minecraft client is not sending valid HTTP headers and Nginx is tossing out the request. But I'm totally at a loss. Any help would be appreciated.
try this in your DNS records
A RECORD
Name one.example.com
Value <server_ip>
TTL 86400
Name two.example.com
Value <server_ip>
TTL 86400
SRV RECORD
Name _minecraft._tcp.one.example.com
Port 25500
Value one.example.com
Name _minecraft._tcp.two.example.com
Port 25501
Value two.example.com
As Dag Nabbit stated, a Minecraft server does not talk http. You would typically do this via NAT. A proxy server needs to know the protocol, because as the name suggests, it acts on behalf of the the client. Nginx knows various protocols, not just http, but Minecraft is not one of them. You can however write a proxy module for this protocol and use the existing nginx infrastructure. Since I'm not familiar with the protocol, I can't comment on the fact that this would have any advantages over NAT.
One thing to note for future readers, while yes nginx does pass connections off as a "proxy" to any server:port listing that is defined though the upstream definition in a socks proxy style of connection. This does not work when nginx itself is listening for HTTP communications. This is simply because nginx is is designed by default as a dead simple static http server.
Any sort of reverse proxing of TCP/UDP connections is more scalable at a lower OSI level (ie layer 3 or layer 2 instead of layer 6/7 as nginx is operating at). This is where Source and Destination NATs come into play which is better handled by a firewall or routing policy directive of your edge device.
DNS-RR is not the best solution as this, while yes lower level OSI layering, is only viable if the end applications (layer 7 OSI) understand the method. Minecraft (or just about any game server) at last check did not have this built into the game's networking code.
Now I did look into this and there is a few solutions for minecraft itself that one should look further into:
Transporter plugin
BungeeCord
Be sure to read all the documentation as these are very complex to configure and install. Hench the recommendation to just use NAT-ed network topology instead.
I tried to setup my multiple minecraft instances with SRV but that also doesn't work
nslookup of my srv records show:
C:\Users\Administrator>nslookup -type=SRV _minecraft._tcp.xxx.net
Server: mijnmodem.kpn
Address: 192.168.1.1
Non-authoritative answer:
_minecraft._tcp.xxx.net SRV service location:
priority = 5
weight = 5
port = 25565
svr hostname = camelot.xxx.net
_minecraft._tcp.xxx.net SRV service location:
priority = 5
weight = 5
port = 25566
svr hostname = cityworld.xxx.net
On my router(ZTE H369) port 25565 and 25566 are straight forwarded (TCP and UDP) to the IP wher the instances run. Accessing the urls (in Minecraft) gives io.netty.channel.Abstart$AnnotatedConnectException
Any suggestions how to investigate further?