I'm working on an http client and I would like to test it on requests that take some time to finish. I could certainly come up with a python script to suit my needs, something about like:
def slow_server(environ, start_response):
with getSomeFile(environ) as file_to_serve:
block = file_to_serve.read(1024);
while block:
yield block
time.sleep(1.0)
block = file_to_serve.read(1024);
but this feels like a problem others have already encountered. Is there an easy way to serve static files with an absurdly low bandwidth cap, short of a full scale server like apache or nginx.
I'm working on linux, and the way I've been testing so far is with python -m SimpleHTTPServer 8000 in a directory full of files to serve. I'm equally interested in another simple command line server or a way to do bandwidth limiting with one or a few iptables commands on tcp port 8000 (or whatever would work).
The solution I'm going with for now uses a "real" webserver, but a much easier to configure one, lighttpd. I've added the following file to my path (its in ~/bin)
#! /usr/sbin/lighttpd -Df
server.document-root = "/dev/null"
server.modules = ("mod_proxy")
server.kbytes-per-second = env.LIGHTTPD_THROTTLE
server.port = env.LIGHTTPD_PORT
proxy.server = ( "" => (( "host" => "127.0.0.1", "port" => env.LIGHTTPD_PROXY )))
Which is a lighttpd config file that acts as a reverse proxy to localhost; source and destination ports, as well as a server total maximum bandwidth are given as environment variables, and so it can be invoked like:
$ cd /path/to/some/files
$ python -m SimpleHTTPServer 8000 &
$ LIGHTTPD_THROTTLE=60 LIGHTTPD_PORT=8001 LIGHTTPD_PROXY=8000 throttle.lighttpd
to proxy the python file server on port 8000 with a low 60KB per second on port 8001. Obviously, lighttpd could be used to serve the files itself, but this little script can be used to make any http server slow
On Windows you can use Fiddler which is a HTTP proxy debugging tool to simulate very slow speeds. Maybe a similar tool exists on what ever OS you are using.
I remember I once had the same question and my search turned up an Apache2 module that goes by the name of mod_bw (mod_bandwith that is). It served me well for my testings.
Related
There are various one-liner HTTP server commands, e.g. the best-known is probably python -m http.server. I'm looking for a similar command which would run a server that ignores the file path and send all paths to a specific file, e.g. if you visit /foo or /bar, it will serve both from index.html.
And ideally relying on as little installation hassle as possible for a typical Linux/MacOS machine. (e.g. python and http.server will come out of the box to many users.)
It's the same functionality offered by the htaccess rule RewriteRule (.*) /index.html, but without needing to setup Apache. Not sure if any of those one-liner servers support something similar to it, like a command-line argument that would declare the default file for all paths.
Using php, there is a built in development server from the command line, which is super useful.
First example, in the current folder, serving only the file index.html at 127.0.0.1, port 8080:
php -S 127.0.0.1:8080 index.html
Output
PHP 7.2.24-0ubuntu0.18.04.1 Development Server started at Mon Dec 23 15:37:03 2019
Listening on http://127.0.0.1:8080
Document root is /home/nvrm
Press Ctrl-C to quit.
On this case, only the file index.html will respond at http://127.0.0.1:8080
Any http calls on this port, will be redirected to index.html.
Second example binding the whole current folder to localhost, port 5555:
php -S localhost:5555
Output:
PHP 7.2.24-0ubuntu0.18.04.1 Development Server started at Mon Dec 23 09:59:44 2019
Listening on http://localhost:5555
Document root is /home/nvrm
Press Ctrl-C to quit.
This will serve index.html at the adresss http://localhost:5555
If a file index.php exist, then it will be served first (interpreted as php)
All others files in the (sub)folder(s) are served, example http://localhost:5555/css/style.css will respond as well, if this folder and file exist of course. (Otherwise respond error 404)
Third example, to run from anywhere, pass in a path as third param. Using a local ip is also possible, by doing so, the files are available on the whole local network.
Example local ip: 192.168.1.23.
To retrieve our local ip, we can use ifconfig.
php -S 192.168.1.23:8080 ~/www
This will serve the folder www in the home folder on the port 8080: http://192.168.1.23:8080 to everyone on the network.
Obviously, we can run many servers on many different ports in parallel^
Very useful to dev, but also to quickly share files between virtual machines, devices, phones etc.
Alternatively. Listen to all interfaces by using 0.0.0.0 as ip adress. In some cases, this is the sole command that serve well across every devices in the local network.
php -S 0.0.0.0:5555
And then use the local ip as url: http://192.168.1.23:5555
To be able to close the terminal, but to keep the server running, we can use nohup:
nohup php -S localhost:8080 &
Then to kill it, quickly:
fuser -k 8080/tcp
Last example, using a hostname.
To retrieve the machine hostname from the console, the unix command is hostname.
php -S $(hostname):9999
Will bind to something like http://<session_name>-<machine_name>:9999
It is possible to install only the cli version of php to run this (~4mo). It's included in the core.
sudo apt install php-cli
For more advanced server usages, yet simple to configure, warmly, I recommend caddy server
https://github.com/svenstaro/miniserve
to serve only index.html you just do miniserve index.html. It's written in Rust so you don't need any additional dependencies.
#!/usr/bin/env node
const express = require('express');
const server = express();
server.all('/*', (_, res) => {
// You would probably not want to hard-code this,
// but make it a command line argument.
res.sendFile(__dirname + '/index.html');
});
const port = 8000;
server.listen(port, () => {
console.log('Server listening on port', port);
});
Make the file executable (chmod +x) and save it somewhere within your PATH.
I'm attempting to re-run a state from another state. I'm not using watch or watch_in etc b/c i want it to run each time. I configure all my nginx virtual hosts and then at the end another state runs called nginx-certs the relevant portion is here:
nginx-frontend:
module.run:
- name: state.sls
- mods:
- nginx-frontend
During the highstate i see the state_id is executed but has no comments, nor shows it reruns that state, it just completes as Result: True. I can then jump to the salt master and run
sudo salt webserver state.sls nginx-certs
and when it hits nginx-frontend, it does reload all of the virtual hosts, putting the new cert in the config.
I'm curious why this does not run in the highstate.
I have attempted ll sorts of different variations of the simple block outlined above. This one works, but not in the highstate, which is my goal to fix.
If you wonder why i do it this way, all certificates for production and staging terminate at HAProxy and nginx only serves up 80/http1 81/h2, but when building out dev servers i want to assign the cert directly to the server as it will be public facing. I need to build out the virtual hosts first to get port 80 open which is used for lets-encrypt. Then when the cert is available, update the nginx vhosts listen directive and cert paths.
From what I understand: you have one server which you want temporarily configured with Nginx on port 80, then generate its certificate with letsencrypt, then change Nginx configuration to be on port 443.
What you can do is:
have one state which installs and configures Nginx to listen on port 80
have another state with installs/configures/runs letsencrypt
a third state which configures Nginx as you want it to be at the end [1]
you just include them in salt to be run in the specific order like
# custom_nginx.sls
include:
- temp_nginx_on_port_80
- letsencrypt_cert
- nginx
[1] for this I think its better to use formula like the one from the community https://github.com/saltstack-formulas/nginx-formula/ and configure it with pillar data. Obviously if you use it for step 3, you won't be able to use for step 1 (or at least I don't see right now how)
I'm using vagrant (VVV actually) to run local wordpress installs. I want to test different behaviors for different GEO's on my local machine instead of upload it every time to the server which is annoying.
So, I've tried to install the GeoIP nginx module to the local machine with the following guide https://piwik.org/faq/how-to/faq_166/ (and a bit more google but it doesn't matter at the moment).
When I'm using ./configure the following is exists:
checking for GeoIP library ... found
checking for GeoIP IPv6 support ... found
I've also set the .dat files in my conf file, and set the $_SERVER (fastcgi_param) parameters - so they displayed when I'm printing the $_SERVER var.
But those GeoIP vars are empty. I'm not sure about the reason, but 2 things is bothering me. First, when I'm write nginx -V in the terminal the argument --with-http_geoip_module is missing. Second, could it actually works if the REMOTE_ADDR (IP) is not my real IP? (192.168.1.50 for example).
nginx is a bit strange for me, so sorry if something isn't exact..
--
Operating system - macOS, nginx version - 1.3.15, running with VVV (vagrant box)
If there is a reverse proxy in front of your nginx, use geoip_proxy to set IPs whose X-Forwarded-For-Header can be trusted.
You can also use that without actually having a reverse proxy when you're developing. Add your local IP to the geoip_proxy-list and set the X-Forwarded-For-Header to your public IP in your browser (use a plugin like Modify Headers).
I have 2 meteor apps running on one Ubuntu server on DO. I have also set up nginx for "servering"
Config files:
sailsadria.conf : http://pastebin.com/eCicpNxK
ytp.klancir.work.conf : http://pastebin.com/cNKtA0dV
Now...
http://sailsadria.com/ which is on port 3000 works smoothly as expected while http://ytp.klancir.work/ goes on ngnix root. On the other hand http://ytp.klancir.work:3010 goes to the right app that is working on that port (but I suppose that any URL or the IP i forward with the appended port will end up on the right location)
Symlinks are also set up
The domains are configured:
sailsadria: http://screencast.com/t/iqKUlQlDgj8
ytp.klancir.work: http://screencast.com/t/DJJdLfqna
I dont know how to set up that http://ytp.klancir.work/ goes directly to port 3010 in other words - directly to the app...
The SOLUTION: sudo service nginx restart....
I've set up a Flask application to run on a tornado server backed by nginx. I've written a couple of bash scripts to reload server configuration when a new version is deployed, but I am unhappy with them. Basically what I have is:
to start the server (assuming in project root)
# this starts the tornado-flask wrapper
python myapp.py --port=8000 # .. some more misc settings
# this starts nginx
nginx
to stop it
pkill -f 'myapp.py'
nginx -s stop
to restart
cd $APP_ROOT
./script/stop && ./script/start
Many times these don't work smoothly and I need to manually run the commands. Also, I'm looking for a way to verify the service is alive, and start it up if it's down. Thoughts? Thanks.
Supervisor is what you are looking for.
It's what I use to manage my Tornado apps along with some other processing daemons.
It will daemonize, handle logging, pid files... Pretty much everything you need.