I'm trying to serve static contents through varnish(client<-varnish<-nginx) but I'm having an issue accessing it through public IP. Varnish is working fine when accessing it locally (where varnish is running) but throws status 404 when accessing it through public IP.
My flow looks like this:
Client--> Caching_Server[NGINX(used for SSL support only) -> Varnish] --> Origin_Server[Nginx]
My varnish server config is as simple as:
probe healthcheck {
.url = "http://10.10.10.3/healthcheck";
.timeout = 2s;
.interval = 30s;
.window = 5;
.threshold = 3;
}
backend default {
.host = "10.10.10.3";
.port = "80";
.probe = healthcheck;
}
sub vcl_recv {
if (req.url ~ ".ts$") {
unset req.http.Cookie;
}
set req.backend_hint = default;
}
I also checked the Nginx logs and I see that it's throwing 404 which explains why varnish is responding with 404. But my question is why is it working when I test it locally using localhost.
The output from varnishlog -g request -q "ReqUrl eq '/'" will probably give you the answer to your question.
Run the command while sending a request internally, and a request from the internet. Compare the output, and see where there's a difference.
If you need help, add the logs of both requests to your question and I'll help you examine the situation.
Related
I have two servers:
NGINX (it exchanges file id to file path)
Golang (it accepts file id and return it's path)
Ex: When browser client makes request to https://example.com/file?id=123, NGINX should proxy this request to Golang server https://go.example.com/getpath?file_id=123, which will return the response to NGINX:
{
data: {
filePath: "/static/..."
},
status: "ok"
}
Then NGINX should get value from filePath and return file from the location.
So the question is how to read response (get filePath) in NGINX?
I assume you are software developer and your have full control over your application so there is no need to force square peg in a round hole here.
Different kinds of reverse proxies support ESI(Edge Side Includes) technology which allow developer to replace different parts of responce body with content of static files or with response bodies from upstream servers.
Nginx has such technology as well. It is called SSI (Server Side Includes).
location /file {
ssi on;
proxy_pass http://go.example.com;
}
Your upstream server can produce body with content <!--# include file="/path-to-static-files/some-static-file.ext" --> and nginx will replace this in-body directive with content of the file.
But you mentioned streaming...
It means that files will be of arbitrary sizes and building response with SSI would certainly eat precious RAM resources so we need a Plan #B.
There is "good enough" method to feed big files to the clients without showing static location of the file to the client.
You can use nginx's error handler to server static files based on information supplied by upstream server.
Upstream server for example can send back redirect 302 with Location header field containing real file path to the file.
This response does not reach the client and is feed into error handler.
Here is an example of config:
location /file {
error_page 302 = #service_static_file;
proxy_intercept_errors on;
proxy_set_header Host $host;
proxy_pass http://go.example.com;
}
location #service_static_file {
root /hidden-files;
try_files $upstream_http_location 404.html;
}
With this method you will be able to serve files without over-loading your system while having control over whom do you give the file.
For this to work your upstream server should respond with status 302 and with typical "Location:" field and nginx will use location content to find the file in the "new" root for static files.
The reason for this method to be of "good enough" type (instead of perfect) because it does not support partial requests (i.e. Range: bytes ...)
Looks like you are wanting to make an api call for data to run decision and logic against. That's not quite what proxying is about.
The core proxy ability of nginx is not designed for what you are looking to do.
Possible workaround: extending nginx...
Nginx + PHP
Your php code would do the leg work.
Serve as a client to connect to the Golang server and apply additional logic to the response.
<?php
$response = file_get_contents('https://go.example.com/getpath?file_id='.$_GET["id"]);
preg_match_all("/filePath: \"(.*?)\"/", $response, $filePath);
readfile($filePath[1][0]);
?>
location /getpath {
try_files /getpath.php;
}
This is just the pseudo-code example to get it rolling.
Some miscellaneous observations / comments:
The Golang response doesn't look like valid json, replace preg_match_all with json_decode if so.
readfile is not super efficient. Consider being creative with a 302 response.
Nginx + Lua
sites-enabled:
lua_package_path "/etc/nginx/conf.d/lib/?.lua;;";
server {
listen 80 default_server;
listen [::]:80 default_server;
location /getfile {
root /var/www/html;
resolver 8.8.8.8;
set $filepath "/index.html";
access_by_lua_file /etc/nginx/conf.d/getfile.lua;
try_files $filepath =404;
}
}
Test if lua is behaving as expected:
getfile.lua (v1)
ngx.var.filepath = "/static/...";
Simplify the Golang response body to just return a bland path then use it to set filepath:
getfile.lua (v2)
local http = require "resty.http"
local httpc = http.new()
local query_string = ngx.req.get_uri_args()
local res, err = httpc:request_uri('https://go.example.com/getpath?file_id=' .. query_string["id"], {
method = "GET",
keepalive_timeout = 60,
keepalive_pool = 10
})
if res and res.status == ngx.HTTP_OK then
body = string.gsub(res.body, '[\r\n%z]', '')
ngx.var.filepath = body;
ngx.log(ngx.ERR, "[" .. body .. "]");
else
ngx.log(ngx.ERR, "missing response");
ngx.exit(504);
end
resty.http
mkdir -p /etc/nginx/conf.d/lib/resty
wget "https://raw.githubusercontent.com/ledgetech/lua-resty-http/master/lib/resty/http_headers.lua" -P /etc/nginx/conf.d/lib/resty
wget "https://raw.githubusercontent.com/ledgetech/lua-resty-http/master/lib/resty/http.lua" -P /etc/nginx/conf.d/lib/resty
I'm trying to make an http request using lua-resty-http.
I created a simple get api in https://requestb.in
I can make a request using the address: https://requestb.in/snf2ltsn
However, when I try to do this in nginx I'm getting error no route to host
My nginx.conf file is:
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
lua_package_path "$prefix/lua/?.lua;;";
server {
listen 8080;
location / {
resolver 8.8.8.8;
default_type text/html;
lua_code_cache off; #enables livereload for development
content_by_lua_file ./lua/test.lua;
}
}
}
and my Lua code is
local http = require "resty.http"
local httpc = http.new()
--local res, err = httpc:request_uri("https://requestb.in/snf2ltsn", {ssl_verify = false,method = "GET" })
local res, err = httpc:request_uri("https://requestb.in/snf2ltsn", {
method = "GET",
headers = {
["Content-Type"] = "application/x-www-form-urlencoded",
}
})
How can I fix this Issue?
Or is there any suggestion to make http request in nginx?
any clue?
PS: There is a commented section in my Lua code. I also tried to make a request using that code but nothing happened.
Change the package_path like:
lua_package_path "$prefix/resty_modules/lualib/?.lua;;";
lua_package_cpath "$prefix/resty_modules/lualib/?.so;;";
By default nginx resolver returns IPv4 and IPv6 addresses for given domain.
resty.http module uses cosocket API.
Cosocket's connect method called with domain name selects one random IP address You are not lucky and it selected IPv6 address. You can check it by looking into nginx error.log
Very likely IPv6 doesn't work on your box.
To disable IPv6 for nginx resolver use directive below within your location:
resolver 8.8.8.8 ipv6=off;
I'm attempting to get Varnish to cache two different domains with blogs, but upon adding the second one, the previous one stops working,
The basic setup is as following:
backend default {
.host = "127.0.0.1";
.port = "81";
}
backend onedomain {
.host = "127.0.0.1";
.port = "81";
}
backend newdomain {
.host = "127.0.0.1";
.port = "81";
}
acl purge {
"localhost";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
#Bypass large files
if (req.http.x-pipe-mark && req.restarts > 0) {
return(pipe);
}
# all domains in here will return a "pass" which means they won't be cached
if (req.http.host ~ "(www\.)?(domain1.com|domain2.com)") {
return (pass);
}
#else check if something we're going to cache
else if(req.http.host ~ "(www\.)?(onedomain.nu)") {
set req.http.host = "onedomain.com";
set req.backend_hint = onedomain;
}
else if(req.http.host ~ "(www\.)?(newdomain.com)") {
set req.http.host = "newdomain.com";
set req.backend_hint = newdomain;
}
else {
return (pass);
}
Newdomain loads fine while domain4 just sends me to an infinite redirect loop (according to the chrome error)
I added the full config in a pastebin: http://pastebin.com/J1Hb76dZ
I realize Varnish doesn't send any redirect commands itself, the site works on the old configuration, it's only when I try this that the redirect issue arises on one of the websites.
Is there anyone that has experience with this happening and can suggest what to do?
Old question, but try modifying the vcl_hash subroutine. This worked for me on a single site, that included multiple redirects for http -> https and domain.com -> www.domain.com. It should also configure the hash to tell your different domains apart, as that was necessary for me to store all the redirects separate from the site data that caused the dreaded "too many redirects" errors. You may need to adjust/remove the X-Forwarded-Proto header as I am behind a load balancer.
sub vcl_hash {
hash_data(req.http.host);
hash_data(req.url);
hash_data(req.http.X-Forwarded-Proto);
return(hash);
}
I am trying to reverse proxy several socket.io websockets and also rewrite the URLs so that I can have a front end server that can connect to several backend tty.js instances.
For example:
http://mysite.com/server1 reverse proxies to http://server1/ which is running tty.js
http://mysite.com/server2 reverse proxies to http://server2/ which is running tty.js
For HTTP any old proxy works fine. However when I rewrite the URLs instead of the websocket looking like:
http://mysite.com/server1/socket.io/1/?t=1354135029745 -> http://server1/socket.io/1/?t=1354135029745
it looks like:
http://mysite.com/socket.io/1/?t=1354135029745
This of course goes unhandled. I assume the rewrite happens and so the socket assumes it's "connection" is at /socket.io/1/?t=1354135029745 instead of /server1/socket.io/1/?t=1354135029745
Is there some way to do URL rewriting as well as proxy a websocket?
So far I have tried both Varnish, node-http-proxy and Nginx using the TCP proxy module and I haven't been able to figure out how to get this to work.
Right now my Varnish config looks like:
backend backend1 {
.host = "1.1.1.1";
.port = "8080";
.connect_timeout = 1s;
.between_bytes_timeout = 60s;
.max_connections = 800;
}
backend backend2 {
.host = "2.2.2.2";
.port = "8080";
.connect_timeout = 1s;
.between_bytes_timeout = 60s;
.max_connections = 800;
}
sub vcl_recv {
set req.grace = 120s
if(req.url ~ "^/server1/") {
set req.url = regsub(req.url, "^/server1/", "/");
set req.backend = backend1;
return(pipe);
}
if(req.url ~ "^/server2/") {
set req.url = regsub(req.url, "^/server2/", "/");
set req.backend = backend2;
return(pipe);
}
}
sub vcl_pipe {
if(req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
} else {
set bereq.http.connection = "close";
}
return(pipe);
}
I am forced to leave out the obligatory:
if(req.url ~ "^/socket.io/") {
set req.backend = backend1;
return(pipe);
}
since I have multiple backends. Is there a way to get this to work using node-http-proxy, Nginx or Varnish? Or if none of those have the ability, would something like haproxy or really anything else be able to?
My first inclination would be to set a custom header so when the URL does get rewritten it knows which backend it should hit, but I'm not sure if/how that would work.
Edit 1: I was able to sort of fix this issues by also pulling the req.http.referer and adding that to the socket.io statement:
if(req.url ~ "^/socket.io/" && req.http.referer ~ "server1"){
set req.backend = backend1;
return(pipe);
}
However it appears to be a little wonky and seems to revert socket.io to xhr polling which is not ideal but better than nothing.
Edit 2: After testing it out with a few backends what happens is once long polling starts it breaks the rest of the proxy. As soon as I refresh the page now the reverse proxy instead of going:
http://mysite.com/server1 -> http://server1
goes:
http://mysite.com/ -> http://server1
until I reload varnish and then goes back.
Most likely you also need to change the Host: header as well in your vcl_recv():
sub vcl_recv {
set req.grace = 120s
if(req.url ~ "^/server1/") {
set req.url = regsub(req.url, "^/server1/", "/");
set req.http.host = "server1";
set req.backend = backend1;
return(pipe);
}
if(req.url ~ "^/server2/") {
set req.url = regsub(req.url, "^/server2/", "/");
set req.http.host = "server2";
set req.backend = backend2;
return(pipe);
}
}
That should take care of your initial problem.
However using pipe in Varnish has it's pitfalls which you might want to consider. The most important of which is the lack of X-Forwarded-For header after the first request.
I've apache2 and nginx. I set "trust proxy headers" to true in configuration, but anyway get internal ip when calls $request->getClientIp(); What do I wrong?
If I calling getClientIp with parameter $proxy = true then I getting correct IP. But there is configuration where proxy headers enabled, aren't that enough?
Actually this is an issue, fixed by this merge https://github.com/symfony/symfony/commit/40599ec0a24e688ef5903e2bd3cfb29b5ab29a18
In short: you always need to use $proxy = true if you're planning on using some kind of reverse proxy. With this parameter set (and trustProxyData(); enabled), $this->getClientIp(); will return the correct IP with reverse proxy.
Explanation: even after configured, proxy headers will return HTTP_X_FORWARDED_FOR or HTTP_CLIENT_IP as the user IP, while REMOTE_ADDR will return server localhost address (most likely 127.0.0.1). $proxy = true checks exactly that. Here's the source code for this function:
public function getClientIp($proxy = false)
{
if ($proxy) {
if ($this->server->has('HTTP_CLIENT_IP')) {
return $this->server->get('HTTP_CLIENT_IP');
} elseif (self::$trustProxy && $this->server->has('HTTP_X_FORWARDED_FOR')) {
return $this->server->get('HTTP_X_FORWARDED_FOR');
}
}
return $this->server->get('REMOTE_ADDR');
}