Tornado run test with nginx proxy - nginx

I use Tornado and write some tests. And its everything fine.
Then I have used nginx for proxy:
server {
listen 80;
server_name mine.local;
location / {
proxy_pass http://localhost:8000;
}
}
It work nice. But.
In tests I use AsyncHTTPTestCase and get_app method, which returns Application.
The problem is: tests "looks" on default 127.0.0.1:8000 - Tornado starts on the port 8000, and all self.app.reverse_url('name') returns 127.0.0.1:8000/path.
But I need, that all requests from tests go to nginx (proxy):
mine.local/path
In hosts I have:
mine.local 127.0.0.1
In nginx I use some lua-scripts, that do all dirty-work. So I need, that tests make requests on mine.local, not on default 127.0.0.1:8000.
How to do this?
Thanks!

def bind_unused_port():
"""Binds a server socket to an available port on localhost.
Returns a tuple (socket, port).
"""
[sock] = netutil.bind_sockets(8000, 'localhost', family=socket.AF_INET)
port = sock.getsockname()[1]
return sock, port
class MineTestCase(AsyncHTTPTestCase):
def setUp(self):
super(AsyncHTTPTestCase, self).setUp()
sock, port = bind_unused_port()
self.__port = port
self.http_client = self.get_http_client()
self._app = self.get_app()
self.http_server = self.get_http_server()
self.http_server.add_sockets([sock])
def get_url(self, path):
url = '%s://%s:%s%s' % (self.get_protocol(), 'mine.local',
80, path)
return url

Related

Which directive I can run before ssl_certificate_by_lua_block to get user-agent information in openresty

I am using OpenResty to generate SSL certificates dynamically.
I am trying to find out the user-agent of request before running ssl_certificate_by_lua_block and decide If I want to continue with the request or not.
I found out that ssl_client_hello_by_lua_block directive runs before ssl_certificate_by_lua_block but if I try to execute ngx.req.get_headers()["user-agent"] inside ssl_client_hello_by_lua_block I get the following error
2022/06/13 09:20:58 [error] 31918#31918: *18 lua entry thread aborted: runtime error: ssl_client_hello_by_lua:6: API disabled in the current context
stack traceback:
coroutine 0:
[C]: in function 'error'
/usr/local/openresty/lualib/resty/core/request.lua:140: in function 'get_headers'
ssl_client_hello_by_lua:6: in main chunk, context: ssl_client_hello_by_lua*, client: 1.2.3.4, server: 0.0.0.0:443
I tried rewrite_by_lua_block but it runs after ssl_certificate_by_lua_block
Are there any directive that can let me access ngx.req.get_headers()["user-agent"] and run before ssl_certificate_by_lua_block as well?
My Nginx conf for reference.
nginx.conf
# HTTPS server
server {
listen 443 ssl;
rewrite_by_lua_block {
local user_agent = ngx.req.get_headers()["user-agent"]
ngx.log(ngx.ERR, "rewrite_by_lua_block user_agent -- > ", user_agent)
}
ssl_client_hello_by_lua_block {
ngx.log(ngx.ERR, "I am from ssl_client_hello_by_lua_block")
local ssl_clt = require "ngx.ssl.clienthello"
local host, err = ssl_clt.get_client_hello_server_name()
ngx.log(ngx.ERR, "hosts -- > ", host)
-- local user_agent = ngx.req.get_headers()["user-agent"]
-- ngx.log(ngx.ERR, "user_agent -- > ", user_agent)
}
ssl_certificate_by_lua_block {
auto_ssl:ssl_certificate()
}
ssl_certificate /etc/ssl/resty-auto-ssl-fallback.crt;
ssl_certificate_key /etc/ssl/resty-auto-ssl-fallback.key;
location / {
proxy_pass http://backend_proxy$request_uri;
}
}
If someone is facing the same issue.
Here is the email group of OpenResty that helped me.
I was not thinking correctly. The certificate negotiation happens before a client send user-agent data(that comes in after the SYNACK reaches the client). So you cant save issuing the certificate in the process. Hard luck.
Once the handshake and the Client/Server Hello happens then the server has the user-agent, you can do the blocking under access_by_lua_block.

Redirect gRPC traffic using nginx from HTTPS to HTTP

I am planning to redirect HTTPS and HTTP gRPC traffic using nginx for a special use case. I am being able to recreate the problem using a hello world example. The main documentation I have used are [Introducing gRPC Support with NGINX 1.13.10][1] and [Nginx as Reverse Proxy with GRPC][2].
Firstly, I created certificate files for the ssl connection using
openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt -subj '/CN=localhost'
When I follow the article, I am being able to successfully route traffic from a secure grpc client to a secure grpc server. However, my use case needs to forward traffic from a secure nginx port to an insecure grpc server. The client, nginx.conf and server code attached below.
nginx.conf (Needs to reroute traffic to an insecure port)
upstream dev {
server localhost:1338;
}
server {
listen 1449 ssl http2;
ssl_certificate /ssl/server.crt; #Enter you certificate location
ssl_certificate_key /ssl/server.key;
location /helloworld.Greeter {
grpc_pass grpcs://dev;
}
}
client.py (Includes ssl certificate to hit nginx secure endpoint)
from __future__ import print_function
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
host = 'localhost'
port = 1449
with open('/home/ubuntu/Documents/ludex_repos/nginx-grpc/server.crt', 'rb') as f:
trusted_certs = f.read()
credentials = grpc.ssl_channel_credentials(root_certificates=trusted_certs)
with grpc.secure_channel(f'{host}:{port}', credentials) as channel:
stub = helloworld_pb2_grpc.GreeterStub(channel)
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
print(f"========================Greeter client received: {response.message}===============================")
if __name__ == '__main__':
logging.basicConfig()
run()
server.py (Has insecure port)
from concurrent import futures
import time
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
_ONE_DAY_IN_SECONDS = 60 * 60 * 24
class Greeter(helloworld_pb2_grpc.GreeterServicer):
def SayHello(self, request, context):
return helloworld_pb2.HelloReply(message='Hello, %s!' % request.name)
def serve():
port = '1338'
with open('/ssl/server.key', 'rb') as f:
private_key = f.read()
with open('/ssl/server.crt', 'rb') as f:
certificate_chain = f.read()
server_credentials = grpc.ssl_server_credentials(((private_key, certificate_chain,),))
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
**If I change this to a secure port then it routes traffic correctly via nginx**
#server.add_secure_port('[::]:'+port, server_credentials)
server.add_insecure_port('[::]:'+port)
print("Server Started...")
server.start()
try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
server.stop(0)
if __name__ == '__main__':
logging.basicConfig()
serve()
Secure to secure response
========================Greeter client received: Hello, you!===============================
Secure to insecure response
Traceback (most recent call last):
File "greeter_client.py", line 45, in <module>
run()
File "greeter_client.py", line 39, in run
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
File "/home/ubuntu/anaconda3/envs/fp/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/ubuntu/anaconda3/envs/fp/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Received http2 header with status: 502"
debug_error_string = "{"created":"#1641485952.541123035","description":"Received http2 :status header with non-200 OK status","file":"src/core/ext/filters/http/client/http_client_filter.cc","file_line":132,"grpc_message":"Received http2 header with status: 502","grpc_status":14,"value":"502"}"
>
I understand a reverse proxy is possible and I've seen examples forwarding traffic from https to http using webpages but I'm not sure if it is possible to do it with gRPC traffic?
[1]: https://www.nginx.com/blog/nginx-1-13-10-grpc/
[2]: https://medium.com/nirman-tech-blog/nginx-as-reverse-proxy-with-grpc-820d35642bff
Try using grpc_pass grpc://... instead of grpcs://...
This updated blog post might help: https://www.nginx.com/blog/deploying-nginx-plus-as-an-api-gateway-part-3-publishing-grpc-services/

NGINX read body from proxy_pass response

I have two servers:
NGINX (it exchanges file id to file path)
Golang (it accepts file id and return it's path)
Ex: When browser client makes request to https://example.com/file?id=123, NGINX should proxy this request to Golang server https://go.example.com/getpath?file_id=123, which will return the response to NGINX:
{
data: {
filePath: "/static/..."
},
status: "ok"
}
Then NGINX should get value from filePath and return file from the location.
So the question is how to read response (get filePath) in NGINX?
I assume you are software developer and your have full control over your application so there is no need to force square peg in a round hole here.
Different kinds of reverse proxies support ESI(Edge Side Includes) technology which allow developer to replace different parts of responce body with content of static files or with response bodies from upstream servers.
Nginx has such technology as well. It is called SSI (Server Side Includes).
location /file {
ssi on;
proxy_pass http://go.example.com;
}
Your upstream server can produce body with content <!--# include file="/path-to-static-files/some-static-file.ext" --> and nginx will replace this in-body directive with content of the file.
But you mentioned streaming...
It means that files will be of arbitrary sizes and building response with SSI would certainly eat precious RAM resources so we need a Plan #B.
There is "good enough" method to feed big files to the clients without showing static location of the file to the client.
You can use nginx's error handler to server static files based on information supplied by upstream server.
Upstream server for example can send back redirect 302 with Location header field containing real file path to the file.
This response does not reach the client and is feed into error handler.
Here is an example of config:
location /file {
error_page 302 = #service_static_file;
proxy_intercept_errors on;
proxy_set_header Host $host;
proxy_pass http://go.example.com;
}
location #service_static_file {
root /hidden-files;
try_files $upstream_http_location 404.html;
}
With this method you will be able to serve files without over-loading your system while having control over whom do you give the file.
For this to work your upstream server should respond with status 302 and with typical "Location:" field and nginx will use location content to find the file in the "new" root for static files.
The reason for this method to be of "good enough" type (instead of perfect) because it does not support partial requests (i.e. Range: bytes ...)
Looks like you are wanting to make an api call for data to run decision and logic against. That's not quite what proxying is about.
The core proxy ability of nginx is not designed for what you are looking to do.
Possible workaround: extending nginx...
Nginx + PHP
Your php code would do the leg work.
Serve as a client to connect to the Golang server and apply additional logic to the response.
<?php
$response = file_get_contents('https://go.example.com/getpath?file_id='.$_GET["id"]);
preg_match_all("/filePath: \"(.*?)\"/", $response, $filePath);
readfile($filePath[1][0]);
?>
location /getpath {
try_files /getpath.php;
}
This is just the pseudo-code example to get it rolling.
Some miscellaneous observations / comments:
The Golang response doesn't look like valid json, replace preg_match_all with json_decode if so.
readfile is not super efficient. Consider being creative with a 302 response.
Nginx + Lua
sites-enabled:
lua_package_path "/etc/nginx/conf.d/lib/?.lua;;";
server {
listen 80 default_server;
listen [::]:80 default_server;
location /getfile {
root /var/www/html;
resolver 8.8.8.8;
set $filepath "/index.html";
access_by_lua_file /etc/nginx/conf.d/getfile.lua;
try_files $filepath =404;
}
}
Test if lua is behaving as expected:
getfile.lua (v1)
ngx.var.filepath = "/static/...";
Simplify the Golang response body to just return a bland path then use it to set filepath:
getfile.lua (v2)
local http = require "resty.http"
local httpc = http.new()
local query_string = ngx.req.get_uri_args()
local res, err = httpc:request_uri('https://go.example.com/getpath?file_id=' .. query_string["id"], {
method = "GET",
keepalive_timeout = 60,
keepalive_pool = 10
})
if res and res.status == ngx.HTTP_OK then
body = string.gsub(res.body, '[\r\n%z]', '')
ngx.var.filepath = body;
ngx.log(ngx.ERR, "[" .. body .. "]");
else
ngx.log(ngx.ERR, "missing response");
ngx.exit(504);
end
resty.http
mkdir -p /etc/nginx/conf.d/lib/resty
wget "https://raw.githubusercontent.com/ledgetech/lua-resty-http/master/lib/resty/http_headers.lua" -P /etc/nginx/conf.d/lib/resty
wget "https://raw.githubusercontent.com/ledgetech/lua-resty-http/master/lib/resty/http.lua" -P /etc/nginx/conf.d/lib/resty

nginx lua-resty-http no route to host error

I'm trying to make an http request using lua-resty-http.
I created a simple get api in https://requestb.in
I can make a request using the address: https://requestb.in/snf2ltsn
However, when I try to do this in nginx I'm getting error no route to host
My nginx.conf file is:
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
lua_package_path "$prefix/lua/?.lua;;";
server {
listen 8080;
location / {
resolver 8.8.8.8;
default_type text/html;
lua_code_cache off; #enables livereload for development
content_by_lua_file ./lua/test.lua;
}
}
}
and my Lua code is
local http = require "resty.http"
local httpc = http.new()
--local res, err = httpc:request_uri("https://requestb.in/snf2ltsn", {ssl_verify = false,method = "GET" })
local res, err = httpc:request_uri("https://requestb.in/snf2ltsn", {
method = "GET",
headers = {
["Content-Type"] = "application/x-www-form-urlencoded",
}
})
How can I fix this Issue?
Or is there any suggestion to make http request in nginx?
any clue?
PS: There is a commented section in my Lua code. I also tried to make a request using that code but nothing happened.
Change the package_path like:
lua_package_path "$prefix/resty_modules/lualib/?.lua;;";
lua_package_cpath "$prefix/resty_modules/lualib/?.so;;";
By default nginx resolver returns IPv4 and IPv6 addresses for given domain.
resty.http module uses cosocket API.
Cosocket's connect method called with domain name selects one random IP address You are not lucky and it selected IPv6 address. You can check it by looking into nginx error.log
Very likely IPv6 doesn't work on your box.
To disable IPv6 for nginx resolver use directive below within your location:
resolver 8.8.8.8 ipv6=off;

Serving Pyramid App with UWSGI and NGINX

I am new to NGINX, uWSGI AND Pyramid, and I am trying to serve a Pyramid app through uWSGI using nginx as a reverse proxy. I am really stuck at the moment and am hoping someone can make some suggestions for how to solve this. If you can explain a little what might be going on, that would be helpful too, as my understanding is very limited!
Currently, I am getting an `Internal Server Error' from uWSGI when I visit the reverse proxy URL. In the uWSGI error log, I am getting the error:
--- no python application found, check your startup logs for errors ---
The application works fine when I serve through uWSGI alone, launching with pserve. I can launch it from my virtual envelope as follows:
bin/pserve my-app/uwsgi.ini
But when I start nginx, and visit the proxy address, I get the Internal Server Error.
The settings I have in uwsgi.ini are as follows:
[app:main]
use = egg:myapp
pyramid.reload_templates = true
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
pyramid_debugtoolbar
[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 6543
[loggers]
keys = root, musiccircle
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = ERROR
handlers = console
[logger_musiccircle]
level = ERROR
handlers =
qualname = musiccircle
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s
[uwsgi]
socket = unix://home/usr/env/myapp/myapp.sock
master = true
processes = 48
cpu-affinity = 12
harakiri = 60
post-buffering = 8192
buffer-size = 65535
daemonize = ./uwsgi.log
pidfile = ./pid_5000.pid
listen = 32767
reload-on-as = 512
reload-on-rss = 192
limit-as = 1024
no-orphans = true
reload-mercy = 8
log-slow = true
virtualenv = /home/usr/env
And in the corresponding myapp.conf file in nginx, I have the following:
upstream myapp {
server 127.0.0.1:6543;
}
server {
listen 8080;
server_name myapp.local www.myapp.local;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/usr/env/myapp;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
charset utf-8;
location / {
include uwsgi_params;
uwsgi_pass unix://home/usr/env/myapp/myapp.sock;
}
}
If you need to see anything else, please let me know. As you can see, Nginx is configured to serve at port 8080 (which it does), and the Pyramid app is being served by uWSGI to port 6543 (which it does).
Thanks in advance.
It seems Pyramid projects are intended to be installed (setup.py) and then run with a .ini configuration file with pserve. Pserve then passes in these config file details as **settings to your Pyramid app at run time.
This is different than, say, Flask which is not installed and generally has no configuration file. Such a Flask application can be run by uWSGI as needed, with all run-time configuration being handled by uWSGI or environment variables.
Since Pyramid usually needs a config file at run time, and relies on pserve to provide them when using a config file (ie production.ini), I think you'll have to run uwsgi --ini-paste production.ini (or if running with Pypy, uwsgi --pypy-paste production.ini) (thanks to #Sorrel)

Resources