I have an NGINX Server set up, I'd like to take a request and forward it to another application on a TCP port.
Let's say I have the following JSON payload
{
"someKey1": 1234,
"someKey2": "a string"
}
This is sent inside query parameters like the following
https://mywebsite.com?payload=%7B%0A%20%22someKey1%22%3A%201234%2C%0A%20%22someKey2%22%3A%20%22a%20string%22%0A%7D
Is there a way to forward that JSON payload to TCP port 1234 natively with NGINX?
Additionally, can I do any pre-processing of the above payload prior to it being forwarded to TCP port 1234. For example, I'd like to covert the above JSON to
someKey1=1234,someKey2="a string"
And then forward this data to TCP port 1234
I understang I'd have to create some sort of REST endpoint using something like springboot to do this, but I'd really like to try and accomplish the above natively with NGINX if possible.
Nginx's primary purpose is HTTP server/proxy.
It can be scripted via ngx_http_lua_module, but for your task it is much simpler to make an app/microservice that will listen HTTP and forward your custom protocol, or modify your app that listens mentioned port to understand HTTP.
When your endpoint talks HTTP - nginx can then be used for routing:
location /some_path/ {
proxy_pass http://localhost:1234/;
}
location /some_other_path/ {
proxy_pass http://localhost:1235/;
}
NGINX is simple web-server, which accepts HTTP requests and forwards them to configured location (may be application server, or any other web-server), and responds back on HTTP to the requester. Data can't be processed inside NGINX.
You can configure forwarding rules in default file under sites-available directory in NGINX installation directory.
Here is the nice tutorial of NGINX configuration which might help you.
Related
Good day,
I want to connect to my kafka server from the internet. Kafka installed on the virtual server and all servers hidden behind a nginx.
I updated kafka settings (server.properties).
Added: listeners=PLAINTEXT://:9092
I can connect to kafka server from local network via ip address 10.0.0.1:9092, but unable connect from internet by domain name.
Response from kafka: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic test-topic not present in metadata after 60000 ms.
Nginx: [26/Nov/2019:12:38:25 +0100] "\x00\x00\x00\x14\x00\x12\x00\x02\x00\x00\x00\x00\x00" 400 166 "-" "-" "request_time=1.535" "upstream_response_time=-" "upstream_connect_time=-" "upstream_header_time=-"
nginx conf:
server {
listen 9092;
server_name site.name;
# Max Request size
client_max_body_size 20m;
location / {
proxy_pass http://10.0.0.1:9092;
}
}
Does anyone know what the problem is?
Kafka doesn't use http protocol for communication, so it can't be fronted by an HTTP reverse proxy.
You'll have to use nginx stream definition blocks for TCP proxying
(I've not tried this personally)
https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
unable connect from internet by domain name.
Sounds like an issue with your advertised.listeners configuration. Note that there is no clear way to "hide" Kafka behind a proxy since your clients are required to communicate directly with each broker individually (therefore defeating the purpose of having
Ngnix unless you want to use one Nginx server or open a new port, per broker), and would therefore also require Kafka to know that it would need to "advertise" the proxy rather than its own address.
If you really want to expose Kafka to the public web, you should really be using SSL/SASL listeners, not PLAINTEXT
If you want to use HTTP, then you can install Kafka REST Proxy, then put Nginx in front of that. Then your clients would use http rather than standard kafka libraries
I know I could use some like this:
stream {
upstream ssh {
server X.X.X.X:22;
}
server {
listen 2222;
proxy_pass ssh;
}
}
to proxy pass incoming traffic to port 2222 to another IP's port 22.
Straightforward. But, is there a way to create a dynamic proxy that accepts final destination's hostname and port as parameters?
Something that could be used like this:
proxy_hostname:8080?destination_hostname=example.com&destination_port=1111
ngx_stream_core_module does not accept url parameters. Could nginx be used as a dymanic proxy or only for static tunneling?
I'm asking this because I need a way to hide the IP of a machine firing php mysql requests.
mysqli_connect($hostname, ...)
right now I cannot specify a proxy for the php script alone, only for the entire machine.
Maybe with a small script and fcgiwrap:
https://www.nginx.com/resources/wiki/start/topics/examples/fcgiwrap/
fcgiwrap calls a bash script where you can convert the URI to the program you want to call (mysql) and return the output to nginx as web content.
You could also alter the config of nginx and reload the service. This way you could "dynamicly" open/forward ports. Quite insecure if you make it publicly available.
Theres a server in a customer that runs a nginx, a salt master daemon from saltstack and a secret web app that does secret things.
Considerations:
In this scenario, theres only one ip, only one server and multiple DNS records available;
I have nginx running in port 80;
And salt master running in 6453;
A domain.example.com binding to that IP, exposing my nginx 80 port, that points to the secret webapp;
otherdomain.example.com binding to the same IP, exposing my nginx 80 port, that I want to use to proxy salt port.
That customer has a machine in other place, that does need to connect to the salt and the internet connection is provided by a secret organization and they only allow connections to port 80, no negotiation possible.
My question:
Is possible to use nginx to redirect the otherdomain.example.com 80 port to the 6453 port? I tried the following:
server {
listen 80;
server_name otherdomain.example.com;
proxy_pass 127.0.0.1:6453;
}
But that doesn't work as expected. It is possible? There's some way to do this using nginx?
The error I got from log was:
"proxy_pass" directive is not allowed here
proxy_pass needs to be specified within a location context, and is fundamentally a Web Thing. It only comes into play after the web headers are sent and interpreted.
Things like what you're trying to accomplish are commonly done using HAProxy in tcp mode, although there is a tcp proxy module that also does similar things.
However, I don't think you're going to be very successful, as ZMQ does not participate in the protocol (HTTP Host: headers) that easily allows you to tell the web requests apart from the non-web requests (that come in on the same port).
My recommendation is to either find some way to use another port for this, a second IP address, or write a tricky TCP proxier that'll identify incoming HTTP and/or ZMQ connections and transparently forward them to the correct local port.
I am using an Ubuntu machine with an Ubuntu guest OS. On the guest OS, I ran my OpenDaylight controller, making the topologies with Mininet and viewing them in the OpenDaylight GUI at localhost:8080. Next, I used Postman REST API Client extension on my Chrome Browser to make a GET request to my ODL Controller:
localhost:8080/restconf/operational/opendaylight-inventory:nodes/
I got the proper response to it in XML format. Now, I have to pass my request through NGINX proxy to 3Scale and get authentication using the app_id and app_key parameters. The request is then to be forwarded to the ODL controller so that I gan get the proper response.
I have already downloaded the proxy config files from NGINX. What modifications must be made in these files? What should be the request I enter in the Postman Client to get the same response as before?
You should only need to change the location of the nginx_.lua file in nginx_.conf
If you want to change the port that Nginx listens on, you will also need to change the listen directive in the server block, to your desired port e.g
server {
lua_code_cache off;
listen 81;
Also, you will need to ensure that there is an upstream block for your backend, e.g
upstream backend_localhost {
server localhost:8080 max_fails=5 fail_timeout=30;
}
but if you have entered this in the proxy configuration wizard that should already be there.
That should be all that you need to change/check.
The request in Postman should target Nginx instead of the ODL Controller, and pass in the application credentials e.g if Nginx is running on port 81
localhost:81/restconf/operational/opendaylight-inventory:nodes/?app_id=<YOUR_APP_ID>&app_key=<YOUR_APP_KEY>
Hopefully that should clear up any doubts. However, you can always email us at support#3scale.net if you have any further questions or add any comments here.
I have nginx running on my server, listening port 80 and 433. I know nginx has a number ways of port forwarding that allows me to forward request like: http://myserver:80/subdir1 to some address like: http://myserver:8888.
My question is it possible to configure nginx so that i can forward NON-http request (just those plain TCP connection) to some other port? It's very easy to test if it's a http request because the first bytes will be either "GET" or "POST". Here's the example.
The client connected to nginx .
The client send:
a. HTTP get request: "GET / HTTP 1.1": some rule for HTTP
b. Any bytes that can't be recognized as HTTP header: forward it to some other port, say, 888, 999, etc.
Is it technically possible? Or would you suggest a way to do this?
It is possible since nginx 1.9.0:
http://nginx.org/en/docs/stream/ngx_stream_core_module.html
Something along these lines (this goes on top level of nginx.conf):
stream {
upstream backend {
server backend1.example.com:12345;
}
server {
listen 12345;
proxy_pass backend;
}
}
This is technically possible for sure.
You can modify open source tcp proxies like nginx module called nginx_tcp_proxy_module or HAproxy.
Or you can write a nginx module similar to above one to do this for you.
if nginx remote proxying with HTTP, your client could use the HTTP CONNECT command, then it connects with the remote port and forwards all data as "raw" (or at least I think so).