My localhost running on http://localhost:8080. Now, I have a requirement like this, whenever I type http://www.mywebsite.com, it should load my localhost and if I type https://www.mywebsite.com, it should load the live website.
To achieve this I tried the hosts(/etc/hosts) file and Nginx but it also stops loading the live website in my system.
Host file content:
127.0.0.1 www.mywebsite.com
nginx config
server {
listen 80;
server_name www.mywebsite.com;
location / {
proxy_pass http://127.0.0.1:8080;
}
}
Completely agree with the other answers, mapping from nginx on a remote host to your localhost can be difficult unless you know the public IP address of your local machine, ideally it should be static.
Alternatives
I would encourage giving a try to some proxy tools that can be installed on your local machine, i.e. Charles Proxy and its Map Remote feature.
Once installed, follow these steps:
Install and trust the root certificate Help -> SSL Proxying -> Install Charles Root Certificate
Enable Map Remote feature Tools -> Map Remote -> [x] Enable Map Remote
Add a new rule, e.g. http://www.mywebsite.com -> http://localhost:8080
Now you're ready to test:
Navigate to http://www.mywebsite.com (you should see results from your localhost, proxy took over)
Navigate to https://www.mywebsite.com (you should see results from your remote server)
Map Remote - Rule
Map Remote - Result
You need several pieces to make this work. Thinking through the steps of how a request could be handled:
DNS for www.mywebsite.com points to a single IP, there's no way around that. So all requests for that host, no matter the protocol, will come in to the machine with that IP, the public server.
So we need to route those requests, such that a) https requests are handled by nginx on that same machine (the public server), and b) http requests are forwarded to your local machine. nginx can do a) of course, that's a normal config, and nginx can also do b), as a reverse proxy.
Now the problem is how to route traffic from the public server to your local machine, which is probably at home behind a dynamic IP and a router doing NAT. There are services to do this but to use your own domain is usually a paid feature (eg check out ngrok, I guess Traefik probably handles this too, not sure). To do it yourself you can use a reverse SSH tunnel.
To be clear, this routes any request for http://www.mywebsite.com/ to your local machine, not just your own requests. Everyone who visits the http version of that site will end up hitting your local machine, at least while the tunnel is up.
For 1, you just need your DNS set up normally, with a single DNS record for www.mywebsite.com. You don't need any /etc/hosts tricks, remove those (and maybe reboot, to make sure they're not cached and complicating things).
For 2, your nginx config on the public server would look something like this:
# First the http server, which will route requests to your local machine
server {
listen 80;
server_name www.mywebsite.com;
location / {
# Route all http requests to port 8080 on this same server (the
# public server), which we will forward back to your localhost
proxy_pass http://127.0.0.1:8080;
}
}
# Now the https server, handled by this, public server
server {
listen 443 ssl;
server_name www.mywebsite.com;
# SSL config stuff ...
# Normal nginx config ...
root /var/www/html
location / {
# ... etc, your site config
}
}
The nginx config on your local machine should just be a normal http server listening on port 8080 (the port you mentioned it is running on). No proxying, nothing special here.
For 3), lastly, we need to open a tunnel from your local machine to the public server. If you are on Linux, or macOS, you can do that from the command line with something like this:
ssh user#www.mywebsite.com -nNT -R :8080:localhost:8080 &
If you're on Windows you could use something like PuTTY or the built in SSH client on Win 10.
The important parts of this are (copied from the SSH manpage):
-N Do not execute a remote command. This is useful for just forwarding ports.
-R Specifies that connections to the given TCP port or Unix socket on the remote
(server) host are to be forwarded to the local side.
The -R part specifies that connections to remote port 8080 (where nginx is routing http requests) should be forwarded to localhost port 8080 (your local machine). The ports can be anything of course, eg if you wanted to use port 5050 on your public server and port 80 on your local machine, it would instead look like -R :5050:localhost:80.
Of course the tunnel will fail if your public IP address (on your localhost side) changes, or if you reboot, or your local wifi goes down, etc etc ...
NOTE: you should also be aware that you really are opening your local machine up to the public internet, so will be subject to all the same security risks that any server on the public internet faces, like various scripts probing for vulnerabilities etc. Whenever I use reverse tunnels like this I tend to leave them up only while developing, and shut them down immediately when done (and of course the site will not work when the tunnel is down).
As somebody said above but in different words: I don't really get why you want to access two different locations with basically the same address (different protocols). But dude, who are we to tell you not to do it? Don't let anything or anyone stop you! 😉😁
However, we some times need to think outside the box and come up with different ways to achieve the same result. Why don't you go to your domain provider and set up something like this:
Create a subdomain (check if you need to set an A record for your domain) so you can have something like https://local.example.com/.
Forward the new subdomain to your local IP address (perhaps you need to open/forward ports on you router and install DDClient or a similar service to catch your dynamic local/public IP and send it to your domain provider).
Leave your #/naked record pointing to your website as it is.
Whenever you access: https://www.example.com or http://www.example.com, you'll see your website.
And if you access https://local.example.com or http://local.example.com, you'll access whatever you have on your local computer.
Hope it helps, or at least, gives you a different perspective for a solution.
You have to create or it may be already there in your nginx config files, a section for listen 443 (https).
// 443 is the default port for https
server {
listen 443;
....
}
Whatever solution you pick, it should only work exactly once for you. If you configure your live site correctly, it should do HSTS, and the next time you type "http://www.mywebsite.com" your browser will GET "https://www.mywebsite.com" and your nginx won't even hear about the insecure http request.
But if you really, really want this you can let your local nginx proxy the https site and strip the HSTS headers:
server {
listen 443;
server_name www.mywebsite.com;
proxy_pass https://ip_of_live_server;
proxy_set_header Host $host;
[... strip 'Strict-Transport-Security' ...]
}
Of course you will need your local nginx to serve these TLS sessions with a certificate that your browser trusts. Either adding a self-signed Snake Oil one to your browser, or... since we are implementing bad ideas... add a copy of you live secret key material to your localhost... ;)
You can do this by redirecting HTTP connections on your live site to localhost. First remove the record you have in your hosts file.
Then add the following to your live site's nginx.conf.
server {
listen 80;
server_name www.mywebsite.com;
location / {
# change this to your development machine's IP
if ($remote_addr = 1.2.3.4) {
rewrite ^ http://127.0.0.1:8080;
}
}
}
Good day,
I want to connect to my kafka server from the internet. Kafka installed on the virtual server and all servers hidden behind a nginx.
I updated kafka settings (server.properties).
Added: listeners=PLAINTEXT://:9092
I can connect to kafka server from local network via ip address 10.0.0.1:9092, but unable connect from internet by domain name.
Response from kafka: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic test-topic not present in metadata after 60000 ms.
Nginx: [26/Nov/2019:12:38:25 +0100] "\x00\x00\x00\x14\x00\x12\x00\x02\x00\x00\x00\x00\x00" 400 166 "-" "-" "request_time=1.535" "upstream_response_time=-" "upstream_connect_time=-" "upstream_header_time=-"
nginx conf:
server {
listen 9092;
server_name site.name;
# Max Request size
client_max_body_size 20m;
location / {
proxy_pass http://10.0.0.1:9092;
}
}
Does anyone know what the problem is?
Kafka doesn't use http protocol for communication, so it can't be fronted by an HTTP reverse proxy.
You'll have to use nginx stream definition blocks for TCP proxying
(I've not tried this personally)
https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
unable connect from internet by domain name.
Sounds like an issue with your advertised.listeners configuration. Note that there is no clear way to "hide" Kafka behind a proxy since your clients are required to communicate directly with each broker individually (therefore defeating the purpose of having
Ngnix unless you want to use one Nginx server or open a new port, per broker), and would therefore also require Kafka to know that it would need to "advertise" the proxy rather than its own address.
If you really want to expose Kafka to the public web, you should really be using SSL/SASL listeners, not PLAINTEXT
If you want to use HTTP, then you can install Kafka REST Proxy, then put Nginx in front of that. Then your clients would use http rather than standard kafka libraries
I'm currently running two back end servers on my network and load balancing with Nginx on Windows.
I am load testing the system at the moment however all of my traffic is directed at one server. This is because the ip_hash algorithm sorts traffics by the first 3 octets i.e. 111.222.333.XXX
This is a problem because all of the traffic I am aiming at the server has the same base address (The same first 3 octets) therefore none of my traffic is going to the other server. Does anyone know a way to patch or change the ip_hash algorithm to filter through 4 octets.
Thanks
Nginx open source version supports the hash directive that may work similarly (not exactly the same though) to the sticky session mechanism provided by commercial version:
The generic hash method: the server to which a request is sent is
determined from a user-defined key which may be a text, variable, or
their combination. For example, the key may be a source IP and port,
or URI:
upstream backend {
hash $request_uri consistent;
server backend1.example.com;
server backend2.example.com;
}
https://www.nginx.com/resources/admin-guide/load-balancer/
So how do you use 4 octets from IPv4 with the hash method? Let's find how to get the client IP from the Embedded Variables section http://nginx.org/en/docs/http/ngx_http_core_module.html#variables
$remote_addr client address
So the code looks like:
upstream backend {
hash $remote_addr consistent;
server backend1.example.com;
server backend2.example.com;
}
UPDATE:
If take a look at the Stream module (TCP proxy), the very first example shows exact the same approach:
upstream backend {
hash $remote_addr consistent;
server backend1.example.com:12345 weight=5;
server backend2.example.com:12345;
server unix:/tmp/backend3;
}
server {
listen 12346;
proxy_pass backend;
}
In my environment, elasticsearch sits on a server that only has standard ports (80, 443, etc.) open. All the other ports are firewalled off. I currently have a reverse proxy on port 80 that reroutes all the elasticsearch HTTP requests to elasticsearch's http port.
I would also like to reroute TCP requests to elasticsearch's transport port, so that my local client can directly query elasticsearch as a client node. Nginx 1.9.0 recently allowed TCP load balancing, which is what I would like to utilize for this, but I'm having some trouble getting my system to work. Here is my nginx.conf file (removed the HTTP context to isolate the issue):
worker_processes 1;
events {
worker_connections 1024;
}
stream {
server {
listen 80;
proxy_pass 127.0.0.1:9300;
}
}
My client node is set up to talk to mydomain.com:80, so it should ideally be routing all traffic to the internal transport port. However, I am getting a the following exceptions: org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available
Is there something else I need to configure on my client node or the tcp proxy?
EDIT 1:
Some additional information. I changed Elasticsearch's transport port from 9300 to 8030, which is a port that is open. When I correspondingly changed my nginx.conf to proxypass to 127.0.0.1:8030 my local client node started working, and got appropriate responses to my queries.
So the issue seems to be that if I'm proxy pass to an already open port, it works, but if the port is closed (9300), the proxy pass fails. Does anyone know why this would be and how to fix it? I'd prefer to stick to using port 9300 if possible.
Theres a server in a customer that runs a nginx, a salt master daemon from saltstack and a secret web app that does secret things.
Considerations:
In this scenario, theres only one ip, only one server and multiple DNS records available;
I have nginx running in port 80;
And salt master running in 6453;
A domain.example.com binding to that IP, exposing my nginx 80 port, that points to the secret webapp;
otherdomain.example.com binding to the same IP, exposing my nginx 80 port, that I want to use to proxy salt port.
That customer has a machine in other place, that does need to connect to the salt and the internet connection is provided by a secret organization and they only allow connections to port 80, no negotiation possible.
My question:
Is possible to use nginx to redirect the otherdomain.example.com 80 port to the 6453 port? I tried the following:
server {
listen 80;
server_name otherdomain.example.com;
proxy_pass 127.0.0.1:6453;
}
But that doesn't work as expected. It is possible? There's some way to do this using nginx?
The error I got from log was:
"proxy_pass" directive is not allowed here
proxy_pass needs to be specified within a location context, and is fundamentally a Web Thing. It only comes into play after the web headers are sent and interpreted.
Things like what you're trying to accomplish are commonly done using HAProxy in tcp mode, although there is a tcp proxy module that also does similar things.
However, I don't think you're going to be very successful, as ZMQ does not participate in the protocol (HTTP Host: headers) that easily allows you to tell the web requests apart from the non-web requests (that come in on the same port).
My recommendation is to either find some way to use another port for this, a second IP address, or write a tricky TCP proxier that'll identify incoming HTTP and/or ZMQ connections and transparently forward them to the correct local port.