[NGINX]Can't get proxy_pass and robots.txt to work - nginx

I'm running Kibana behind NGINX and I want to add a robots.txt. I added robots.txt to /var/www/example.com but no matter what I try, when I access example.com/robots.txt it redirects to Kibana.
server {
server_name example.com www.example.com;
location = /robots.txt {
root /var/www/example.com/;
}
location / {
if ($request_uri ~* "^/robots.txt") {
rewrite ^/robots.txt /robots.txt last;
}
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
~
}
server {
if ($host = www.example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name example.com www.example.com;
return 404; # managed by Certbot
}
I've tried the following:
root /var/www/example.com/html;
under location /
location = /robots.txt {
alias /var/www/example/com/html/robots.txt ;
}
under server.
The closest I can get is with the root one, which will redirect example.com/robots.txt to example.com/var/www/example.com/html/robots.txt.

If the path of /robots.txt is /var/www/example.com/html/robots.txt, use:
server {
...
location = /robots.txt {
root /var/www/example.com/html;
}
location / {
...
}
}
The exact match location block will always take precedence. See this document for details.

Related

Unable to redirect from non-www to www with Nginx and Cloudflare

I am super new to Nginx and CF and I have deployed my website to example.com and it is working perfectly and it redirects to https as well. But I want to redirect to www all the time. Right now if I type in example.com it doesn't redirect to www.
I have also set up Cloudflare for my website.
After looking for solutions and found one in Digital Ocean which said to replace this:
server {
if ($host = www.example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
server_name example.com www.example.com;
listen 80;
return 404; # managed by Certbot
}
to this:
server {
if ($host = www.example.com) {
return 301 https://www.example.com$request_uri;
} # managed by Certbot
if ($host = example.com) {
return 301 https://www.example.com$request_uri;
} # managed by Certbot
server_name example.com www.example.com;
listen 80;
return 404; # managed by Certbot
}
But it isn't working. What is wrong here?
Here's the full nginx config for the website:
server {
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name example.com www.example.com;
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $http_cf_connecting_ip;
proxy_set_header X-Forwarded-For $http_x_forwarded_for;
proxy_cache_bypass $http_upgrade;
}
real_ip_header CF-Connecting-IP;
}
server {
if ($host = www.example.com) {
return 301 https://www.example.com$request_uri;
} # managed by Certbot
if ($host = example.com) {
return 301 https://www.example.com$request_uri;
} # managed by Certbot
server_name example.com www.example.com;
listen 80;
return 404; # managed by Certbot
}

How to mask sub-domain using nginx

I have nodejs application working with a main domain, and its connecting thought nginx proxy pass to nodejs application, with redirection rule , as per the redirection rule if request is coming from mobile, it will redirect into a sub-domain, it is working fine.
both main domain and sub-domain are different version ,and running deferent server.
Now I want to mask this sub-domain , I don't want to see my sub-domain in browser, how can I do this.
Below mentioned my nginx conf.
server {
listen 80;
server_name mydomain;
return 301 https://$server_name$request_uri;
}
include mime.types;
server {
listen 443 ssl;
server_name mydomain;
ssl_certificate /etc/ssl/CA_Bundle.ca-bundle;
ssl_certificate_key /etc/ssl/2021/server.key;
location / {
add_header Access-Control-Allow-Origin *;
proxy_set_header Host $host;
proxy_pass http://172.17.0.1:8080;
proxy_redirect off;
expires off;
set $agentflag 0;
if ( $http_user_agent ~* "(Mobile)" ){
set $agentflag 1;
}
if ($request_uri ~ "user-agent"){
set $agentflag 0;
}
if ( $agentflag = 1 ) {
rewrite ^/(.*)/$ /$1 permanent;
return 307 https://mobile.mydomain$request_uri;
}
}
}
server {
listen 80;
server_name mobile.mydomain;
return 301 https://$server_name$request_uri;
}
include mime.types;
server {
listen 443 ssl;
server_name mobile.mydomain;
ssl_certificate /etc/ssl/CA_Bundle.ca-bundle;
ssl_certificate_key /etc/ssl/2021/server.key;
location / {
add_header Access-Control-Allow-Origin *;
proxy_set_header Host $host;
proxy_pass http://172.17.22.1:8081;
proxy_redirect off;
expires off;
}
}
It is working fine now,
I have used proxy_pass , instead of rewrite.
if ( $agentflag = 1 ) {
proxy_pass http://IP-address-mobile-app;

nginx domain with and without wildcard

I have nginx configuration file which has to server example.com and www.example.com.
server {
listen 80;
server_name example.com;
return 301 http://www.example.com$request_uri;
}
server {
listen 80;
server_name www.example.com;
auth_basic "example Login";
auth_basic_user_file /etc/nginx/.htpasswd;
root /projects/www/example;
index index.html;
location ~* \.(html|js|jpg|png|gif|css|perfumes|imgs|map|fonts|otf)$ {
index index.do index.html index.htm;
access_log off;
}
location /.protected {
access_log off;
auth_basic off;
}
location /health {
access_log off;
auth_basic off;
}
location / {
try_files $uri $uri/index.html;
}
location /hello {
proxy_pass http://localhost:8282;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
}
I am trying to protect this domain for now, cuz we are preparing the launch. But there is only link which should let the requests pass through without password, if a link has '.protected' in it.
For example,
www.example.com/.protected/file1.txt should be allowed without entering a password.
It is working ok, but the issue is example.com/.protected/file1.txt (without 'www').
If I type only domain name 'example.com' (without 'www'), it automatically redirects to www.example.com as configured, but 'example.com/.protected/file1.txt' doesn't redirect to 'www.example.com/.protected/file1.txt'. It seems if the domain name has some paths, the domain name (without 'www') doesn't redirect to 'www.example.com'
I am getting 'curl: (6) Could not resolve host:'
Is there anything wrong with my configuration file?

How to turn off ssl for specific url?

Here is my config:
upstream example {
server unix:///home/deploy/apps/example/shared/tmp/sockets/example-puma.sock;
}
server {
listen 80 default_server;
server_name example.org;
location /non_https {
proxy_pass http://example;
}
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl default_server;
ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
include snippets/ssl-params.conf;
root /home/deploy/apps/example/current/public;
try_files $uri/index.html $uri #example;
location #example {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://example;
}
keepalive_timeout 10;
}
I need to turn off ssl for /non_https/* how to get it?
If you place the retuen 301 inside a location "/" block it should work:
server {
listen 80 default_server;
server_name example.org;
location /non_https {
proxy_pass http://example;
}
location / {
return 301 https://$server_name$request_uri;
}
}

NGINX Rewrite to strip a URL Segment

I have moved my site to a new server running nginx from apache.
The old site structure was:
example.com/NEW/
The new structure:
example.com/
I have a few applications that still reference uploads/files like:
example.com/NEW/assets/images/7871862618261826.jpg
I need to rewrite all the requests that include the /NEW/ parameter.
eg:
example.com/NEW/assets/images/7871862618261826.jpg
should be routed to:
example.com/assets/images/7871862618261826.jpg
My Current Nginx config looks like:
server {
listen 80;
server_name www.domain.com domain.com;
return 301 https://www.domain.com$request_uri;
}
server {
listen 443;
if ($http_host = domain.com) {
rewrite (.*) https://www.domain.com$1;
}
location /NEW/ {
rewrite ^/NEW(.*)$ $1 last;
#return 405;
#return 301 https://www.domain.com$request_uri;
}
server_name domain.com www.domain.com;
access_log /var/log/nginx/domaincom.access.log;
error_log /var/log/nginx/domaincom.error.log;
root /var/www/domain.com/public_html/;
index index.html index.htm index.php;
#set default location
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}
How can I achieve the rewrite?
I Finally got it working with the below location block:
location ~ /NEW/.*\.(png|jpg|gif|pdf|doc)$ {
rewrite ^/NEW(.*)$ $1 last;
}
It seems the location block was not being executed as required.

Resources