I want to open any arbitrary TCP socket on a server, but it's behind a proxy and I can only use a port that is intended for HTTP hosting only. Simply put, what is the most transparent way to wrap such a socket into an HTTP connection? Preferably I would call a *nix program through a shell script in the server that would take care of translating the requests.
I apologize if this was answered before, but I am struggling to find and understand anything.
I ended up going with Chisel, which provides a single binary for both servers and clients, with features like authentication and reverse port forwarding. For instance:
chisel server
will run an HTTP server in $PORT or 8080, and
chisel client server.com 4567:123
connects to server.com and maps the remote port 123 into the local port 4567.
Other solutions are still welcome, particularly if they involve more transparency, frequently preinstalled tools like netcat, and if they also provide support for other protocols like UDP.
Related
I am trying to set up a load balancing solution which is able to load balance UDP traffic. In my case, I have several different servers which send UDP package to a load-balancer. Inside each UDP package body, there is a MSG-ID field. Ideally, I want to load-balance UDP traffic to a set of servers based on that MSG-ID. In other words, two UDP packages with same MSG-ID should be sent to the same server (because i want to assemble two UDP packages with same MSG-ID to form a complete package for further processing). But if it's not possible, then a solution based on source/sender IP might be sufficient.
Haproxy doesn't support UDP so I am checking nginx. But seems that only nginx-plus (not free solution) allows you to do load-balance based on ip_hash method?
I wish to know:
What is best open-source solution to help me handle load-balance based on custom MSG-ID inside UDP body.
Does nginx (free/open-source) version support at least UDP load-balance based on ip_hash method.
Is there anyway I can spoof the UDP source/sender IP. Basically, if I have server (such as server A) which receive UDP package from multiple server (server D) and forward it to a set of server (server B, C); I want the server B, C receiving package with IP indicated as server D, not A.
What is drawback of writing a custom layer which read UDP then form a TCP package and then maintain a TCP connection with load balancer to forward that newly-formed TCP package. In that case, both nginx or haproxy can be used.
Thanks.
nginx does support UDP load balancing in both open source nginx(since v1.19.13) and NGINX Plus. Read this post from nginx.com
And nginx provides UDP load-balance based on ip_hash out of box,check the docs
So I think the answer to question No.2 is yes.
Also mentioned in the docs here, nginx upstream module provide hash algorithm on a custom key, which in your case could be MSG-ID, but i don't know how you can get MSG-ID out from you UDP datagram and i guess it won't be easy. I tried to search for a while but got no luck. If you found a solution, hope you can share under this question.
You would ordinarily use proxy_bind $remote_addr transparent; in your server {} context. Note that you need a (very) recent nginx with upstream / udp loadbalancing support for this to work.
i'm having a problem with my app, on a certain situation.
We have a java server with jetty webserver embedded, and an air app on the client side.
It is working properly but on a single situation of a certain customer.
They have a private network that is not administrated by them (and has little chances of being changed as request). So, the only port they allow are 80 and 443.
The communications between the server and the client are through websockets and http.
The "online" check is made through http and, then, we use websockets to notify the client in order to start communication between them.
The thing is, in this situation, the "online" state works properly and any communication send by the client (forced), as it goes through http, gets to the server but, when the server communicates with the client, using websockets, it doesn't work.
We are using wireshark to check the communications: On a working setup, when the client app starts, a websocket is shown on wireshark, on the server side (registering the client on the server). And, after that, websockets that are only used from server to the client, don't show also.
What can be the problem? The port 80? (the same happens with 443 on that network).
Can it be a proxy/firewall that are blocking ws:// messages?
I've read somewhere that wss:// (encripted websockets) would work?
Thanks for your help.
Edit, so, I tried with https and wss communication and the same thing happens.. no websocket is set between the client and server (registering the client on the server).
This situation is happening for http on the customer network. On my test network, it works on http/ws but not with https/wss..
There are many firewalls and gateways out "in the wild" that do not understand the whole WebSocket HTTP/1.1 GET -> UPGRADE -> WebSocket mechanism.
There are several broken firewall implementations will attempt to interpret the WebSocket framing as improper content for HTTP/1.1 (which is a bad reading of the HTTP/1.1 spec) and start to muck with it.
The types of firewalls that inspect/filter/analyze the request/response contents are the ones that seem most susceptible.
I would check that the hardware (or software) that they are using to firewall their network is both compliant and upgraded to support WebSocket RFC-6455.
We have HTTP server , for which we have HTTP client based application (on Linux) working fine.
But now we need to listen on Unix domain sockets from our client application.
So is it possible to send/receive httprequest, httpresponse packet from the unix domain socket?
Scenerio1:When connecting to localhost, it is required to eliminate the SSL
overhead by connecting HTTP to the unix socket instead of HTTPS to the
local port.
Basically Looking for a standard encoding a unix socket path in an HTTP URL.
Many Thanks in advance.
So long as your socket is a stream socket (SOCK_STREAM rather than SOCK_DGRAM) then it's technically possible. There's nothing in HTTP that requires TCP/IP, it just requires a reliable bidirectional stream.
However I've never seen an HTTP client that knows how to connect to such a socket. There's no URL format that I know of that would work, should you actually need to use a URL to talk to the server.
Also note that some things that normal web servers depend on (such as getpeername(), to identify the client) make no sense when you're not using TCP/IP.
EDIT I just saw your edit about mapping localhost to the UNIX socket. This is perfectly feasible, you just need to ensure that the client knows how to find the path of the UNIX socket that should be used instead of connecting to 127.0.0.1:xxx
I want a cloud machine to send a message to a machine behind a corporate NAT / Firewall.
My idea is to install on the corporate machine a client which sends a long HTTP request to the cloud machine and when the cloud has a message it returns the response.
I thought I invented the wheel until I read about "http tunneling" (is this what I am doing?).
I also read that some firewalls block non html traffic even if it is on http.
So what is my chance to make it work?
I have also read that skype uses a more sophisticated machanism.
Is it because my idea does not work or because their idea is faster?
I can compromise on speed now - which approach works and easy to implement?
I know you'd like to do it with TCP/HTTP,
but the way I'd do it is use UDP to
NAT 'hole punch', thus establishing a UDP channel,
and then use UDP packets sent over that
channel as the signaling mechanism...
These may (or may not) be useful or relevant:
http://en.wikipedia.org/wiki/STUN
http://en.wikipedia.org/wiki/Hole_punching
http://en.wikipedia.org/wiki/UDP_hole_punching
http://en.wikipedia.org/wiki/TCP_hole_punching
Also -- if you really have to use HTTP, you could
simply issue a new HTTP request every X seconds...
HTTP Polling, if you will...
If they block non html on port 80, you could try port 443. Unless there is a SSL "man in the middle" proxy (unlikely), you'd be ok.
IIRC skype uses port hopping, so basically you'll need an algorithm to find an unfiltered (by brute force with intelligent guesses) port that you can connect to.
I have web application that runs on Tomcat (and gets HTTP requests) and some other backend standalone application that gets only TCP. For some reasons, I can use outside only port 8080. So, I need to get all TCP requests (from outside) to port 8080 and forward HTTP ones to web application on Tomcat and all TCP pure requests (that are not HTTP) - to standalone application. Internal forwarding could be done to any port, e.g. 8181 on Tomcat and 8282 on standalone application. Is it possible to setup such configuration? How it could be done?
Thanks in advance.
TCP and HTTP are protocols in different networking stack layer. If you want to use some application to filter HTTP requests, your application should deal with Application-Layer information, not Network-Layer(like TCP/UDP).
I don't see how this can be possible generally. You could look packet-by-packet, but the middle of an http body can be arbitary so you can't just look at the data of each packet
If any particular client will send you either http or general TCP but not both, can you do this by source-IP address? Do you know the addresses of either the servers that will send you http requests or the ones that will send you TCP requests?
If you don't know the source IPs, you could heuristically look at the first packet from some previously unknown IP and see if it looks like http, then tag that address as containing http traffic.
What is the content/format ot the TCP communication? Is there any pattern you can detect in that?
Y
Perhaps you could do something like this using iptables + L7 filter. Of course this will only work if you run Linux on your box. Also I don't know how recently l7 filter project has been updated.
Java servlet technology is not limited to Http. The servlet interface lets you read in the incoming input stream via ServletRequest.getInputStream(). So you can create an implementation of Servlet interface and map it in web.xml and you are all set to receive any TCP traffic.
Once you have the read the input stream to sniff the content you will want to forward HTTP requests to an HttpServlet. To do this you will need to make sure that the input stream you pass on is positioned at the very beginning of the input.
EDIT: On reading your question once again, I noticed that you don't plan to expose the Tomcat directly on external port as I originally thought. If you are willing to make the tomcat listen on the external port, you can try the approach described above