i have big problem with encoding on uwsgi/emeror/nginx server.
My app is developed to batch excel files processing.
I use latest version flask and flask-extensions and use flask-excel.
My app runs on Digital Ocean / Ubuntu server and my config files are:
/etc/init/uwsgi.conf
description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]
respawn
env UWSGI=/home/lukas/www/abissk/venv/bin/uwsgi
env LOGTO=/home/lukas/logs/abissk/emperor.log
exec $UWSGI --master --emperor /etc/uwsgi/vassals --die-on-term --uid www-data --gid www-data --logto $LOGTO
/etc/uwsgi/vassals/abissk_uwsgi.ini
[uwsgi]
plugins = python
#pcre = True
#application's base folder
base = /home/lukas/www/abissk/
#enable-threads = true
#python module to import
app = wsgi
module = %(app)
home = %(base)venv
pythonpath = %(base)
#socket file's location
socket = /home/lukas/www/abissk/%n.sock
#permissions for the socket file
chmod-socket = 644
#the variable that holds a flask application inside the module imported at line #6
callable = app
#location of log files
logto = /home/lukas/logs/abissk/%n.log
/home/lukas/www/abissk/wsgi.py
# -*- coding: utf-8 *-*
from app.app import create_app
from app.settings import ProdConfig
app = create_app(config_object=ProdConfig)
/etc/nginx/sites-enabled/abissk_nginx
server {
listen 80;
server_name benela.abis.sk;
charset utf-8;
client_max_body_size 75M;
location / { try_files $uri #yourapplication; }
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/home/lukas/www/abissk/abissk_uwsgi.sock;
}
}
and here is my problem:
When start application with command:
~/www/abissk/venv/bin/uwsgi --ini /etc/uwsgi/vassals/abissk_uwsgi.ini
and upload excel files works all great BUT
when start same aplication with totaly same config files (in /etc/init/uwsgi.conf) with emperor, application works fine, but when upload excel file to batch processing, i see only message: "502 Bad Gateway" and in my log is:
UnicodeEncodeError: 'ascii' codec can't encode character '\xfd' in position 31: ordinal not in range(128)
i was try install uwsgi with apt-get / pip3 install and scenario is same: when run app directly without emperor, works all fine; when run app with emperor (with same configuration), every upload excel file to my app ends with crash :/
Thanks for any answer
add the following lines to your abissk_uwsgi.ini file to enforce uwsgi to use UTF-8.
env LANG=en_US.utf8
env LC_ALL=en_US.UTF-8
env LC_LANG=en_US.UTF-8
Add to uwsgi.ini:
env = LANG=en_US.UTF-8
Only this format helped for me
for Russian language add this in uwsgi.ini
env = LANG=ru_RU.utf8
env = LC_ALL=ru_RU.UTF-8
env = LC_LANG=ru_RU.UTF-8
None of the variables LANG, LC_ALL, LC_LANG helped me.
I fixed the bug only adding this to the uwsgi.ini:
env = PYTHONIOENCODING=UTF-8
Related
I wanted a realtime dashboard and I cloned this https://github.com/roniemartinez/real-time-charts-with-flask.git
I installed the requirements and the application runs normally and fine whenever I run with the "python application.py" command. The "/" route in the application run responsively and fine...
I added the app to a uwsgi ini file and when I run. the web page of the "/" becomes irresponsive and bring the graphs every 30seconds. All works fine with the application. but I think the problem is either my nginx config or uwsgi ini.
what could be wrong ?
[uwsgi]
base = /home/pi/Desktop/Bingo
app = application
module = %(app)
home = %(base)/venv
pythonpath = /home/pi/Desktop/Bingo/venv/bin/python
socket = /home/pi/Desktop/Bingo/%n.sock
chmod-socket = 666
callable = app
logto = /home/pi/Desktop/Bingo/log/uwsgi/%n.log
My nginx config is
server {
listen 80;
server_name localhost;
charset utf-8;
client_max_body_size 75M;
location /static {
root /home/pi/Desktop/Bingo;
}
location / { try_files $uri #labapp;}
location #labapp {
include uwsgi_params;
uwsgi_pass unix:/home/pi/Desktop/Bingo/lab_app_uwsgi.sock;
}
}
The uwsgi logs probably contain some useful clues. Comparing your uwsgi.ini to one of mine, I see a few differences that might be significant. First, I always chdir to the base of the app, which in your case would look like
[uwsgi]
chdir = /home/pi/Desktop/Bingo
home = /home/pi/Desktop/Bingo/venv
module = application:application
... socket
... log
I also wonder if might also have a mismatch, depending on how you've named things, between
socket = /home/pi/Desktop/Bingo/%n.sock
and
uwsgi_pass unix:/home/pi/Desktop/Bingo/lab_app_uwsgi.sock;
The nginx error log is the place to look.
(See https://github.com/davewsmith/nginx-uwsgi-flask-starter/blob/master/provision/starter.ini for a working example)
I have a website running with NGINX + uWSGI + Flask.
The website runs fine most of the time, however every now and then it gets into a state where the pages returned by nginx just show "Internal Server Error". If I look at the uWSGI log when it's doing this, I see the following:
[pid: 1580|app: -1|req: -1/37] 69.162.124.228 () {46 vars in 716 bytes} [Sat May 12 10:25:13 2018] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
It can be in this state for ages, but if I do something seemingly unrelated, like make an arbitrary change to the flask app, then everything will start working again. It will then keep working until some random amount of time later when it starts giving "Internal Server Error" again, without any code changes being made.
I've tried running the uWSGI application directly, and it runs without any error.
I've tried installing Sentry on my flask app to capture any errors, but nothing shows up when this happens.
How do I diagnose this?
What other log files can I look at?
What is likely to be causing this?
I've been at this for over a week now, and read through almost every related question of SO.
I've run out of ideas, and near abandoning this project if I can't figure out what's going on.
Any help would be greatly appreciated.
Here are my files:
uWSGI config (mysite.ini)
[uwsgi]
module = wsgi:app
master = true
processes = 5
socket = mysite.sock
chmod-socket = 660
vacuum = true
die-on-term = true
logto = /var/log/uwsgi/%n.log
wsgi.py
from tunnelling.python.flask_app import app as application
if __name__ == "__main__":
application.run()
nginx:
server {
listen 80;
server_name www.mysite.com;
server_name mysite.com;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/dimraft/mysite/mysite.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/mysite.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/mysite.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
if ($scheme != "https") {
return 301 https://$host$request_uri;
} # managed by Certbot
}
systemd file:
[Unit]
Description=uWSGI instance to serve mysite
After=network.target
[Service]
User=dimraft
Group=www-data
WorkingDirectory=/home/dimraft/mysite
Environment="PATH=/home/dimraft/mysite/mysiteenv/bin"
ExecStart=/home/dimraft/mysite/mysiteenv/bin/uwsgi --ini mysite.ini
[Install]
WantedBy=multi-user.target
maybe your uWSGI config (mysite.ini) error:
change:
module = wsgi:app
to:
module = projectName.wsgi:application
#John was essentially correct on this but, I would like to elaborate for clarity.
I was having the same issue with a project that had essentially the same setup and what fixed it was to move my uwsgi.py module to inside my flask application package like so
myprojectname/
__init__.py
uwsgi.py
then change the module definition in uwsgi.ini to module=myprojectname.uwsgi:application like shown here:
[uwsgi]
module=myprojectname.uwsgi:application
master=true
processes=2
socket=myprojectname.sock
chmod-socket=660
logto=/var/log/uwsgi/uwsgi.log
die-on-term=true
Also to make sure to specify that the callable object is named application as was the case in the original posted code sample. uWSGI defaults to looking for this application named object and I've really struggled to get it to work naming it anything else such as app.
okay sorry this is coming late. i had an error like this too and this is how i solved mine
[uwsgi]
chdir = /home/goodnews/myproject
home = /home/goodnews/myproject/venv
module = wsgi:app
master = true
processes = 5
#socket = myproject.sock
socket = :5000
protocol = http
chmod-socket = 666
vaccum = true
die-on-term = true
in my case i was working on a local virtual machine(running an Ubuntu terminal), that is the reason for
socket = :5000
i was testing a webapp using ssh connection to Ubuntu server.
hopes this helps someone
This issue also happens because the uWSGI python plugin to use is not declared in the .init file. Let's say you have the python3.6 uWSGI python plugin installed, add the following in the .ini file
plugins = python36
Now, you will see other problems or uWSGI will be able to load app 0. Also, make sure that you are in your activated virtual environment to test whether the app is served by uWSGI.
I got this error after making only a small change in the python script.
The cause in my case:
The execute permissions got lost when I copied the new script to the server.
The problem for me is that some files in the website root references outer libs. You need to make sure everything is self-included.
I have an Nginx service that's configured to start automatically on my Windows 10; however, this morning, the service wouldn't start.
The error log says: nginx: [alert] could not open error log file: CreateFile() "C:\someForlderName\build\distribution\.\nginx/logs/error.log" failed (3: The system cannot find the path specified)
Looking at the path in the error log above, I do NOT have the /logs/ folder on my local system so it looks like Nginx doesn't have the proper permissions to create that folder?
I'm setup as an admin user and my service is set to Log On As - Local System Account
This only happens on Windows 10; but the service starts and works on
older Windows i.e 8.1
So does anyone know how to grant administrator's permissions to Nginx so that Nginx can create folders and files on Windows 10 ?
You need:
To install nginx/Windows, download the latest mainline version distribution (1.13.8), since the mainline branch of nginx contains all known fixes. Then unpack the distribution, go to the nginx-1.13.8 directory, and run nginx. Here is an example for the drive C: root directory: (Run cmd as administrator)
cd c:\
unzip nginx-1.13.8.zip
cd nginx-1.13.8
start nginx
Go to: http://localhost:80 -> test install
Goback to console cmd: "nginx -s stop"
Run for next time:
Config with file: "C:\nginx-1.13.8\conf\nginx.conf"
Open cmd as administrator
Run bash: "cd C:\nginx-1.13.8"
Run nginx with bash: "start nginx" . If you run with bash: "nginx", will get trouble for exit nginx.
And
nginx -s stop #fast shutdown
nginx -s quit #graceful shutdown
nginx -s reload #changing configuration, starting new worker processes with a new configuration, graceful shutdown of old worker processes
nginx -s reopen #re-opening log files
Under the directory that you run nginx.exe, try to create a directory named logs, and a file named error.log under log.
It should pass this error.
After downloading zip file, you have unzip.
Make sure that you dont have nested folder names. You have to copy your folder which has nginx.exe file in it, and paste it into C:/ folder.
While running commands, like nginx -s stop, make sure that current your working directory is same as the nginx.exe file.
enter image description here
Nginx start on default port 80, not 8080. Try localhost:80 on browser.
If you want to change port, open C:\nginx-1.16.1\conf\nginx.conf with text editor.
change port number what you want use default port.
server {
listen 80;
server_name localhost;
to:
server {
listen 8080;
server_name localhost;
I had a similar issue with starting the nginx server, but after looking at it closely and trying to run the command in different consoles, I realized it just a simple issue of a missing path.
How I solved it was to cd into the containing folder for the nginx.exe file (which actually contains error logs and all the necessary files) and then run the nginx command which started the server and fixed it for me.
What I am trying to achieve:
I have an nginx web server hosting mydomain.com. When someone types my domain.com into their client I would like my server to serve index.html from /var/www/mydomain/ When they type mydomain.com/flaskapp1 they should see flaskapp1. When they type mydomain.com/flaskapp2 they should see flaskapp2.
I have managed to get one or the other flask apps served using the tutorial here http://www.markjberger.com/flask-with-virtualenv-uwsgi-nginx/ but when trying to implement serving two separate flask apps I run into difficulty. Instead of seeing the flask app I get a 404 message when I try to access either of the flask apps with mydomain.co.uk/flaskapp or mydomain.co.uk/flaskapp2 in a browser.
This is what I have so far:
cat /etc/nginx/sites-available/mydomain.co.uk
server {
listen 80;
server_name www.mydomain.co.uk mydomain.co.uk;
location / {
root /var/www/html/;
index index.html index.htm;
}
location /flaskapp {
include uwsgi_params;
uwsgi_pass unix:/tmp/flaskapp.sock;
}
location /flaskapp2 {
include uwsgi_params;
uwsgi_pass unix:/tmp/flaskapp2.sock;
}
}
The above conf file has been sim linked into /etc/nginx/sites-enabled.
cat /etc/uwsgi/apps-available/flaskapp.ini
[uwsgi]
vhost = true
socket = /tmp/flaskapp.sock
venv = /var/www/flaskapp/venv
chdir = /var/www/flaskapp
module = flaskapp
callable = app
cat /etc/uwsgi/apps-available/flaskapp2.ini
[uwsgi]
vhost = true
socket = /tmp/flaskapp2.sock
venv = /var/www/flaskapp2/venv
chdir = /var/www/flaskapp2
module = flaskapp2
callable = app
Both .ini files have been symlinked into /etc/uwsgi/apps-enabled. UWSGI restarts fine without any issues and is up and running. Both flaskapp.sock and flaskapp2.sock are owned by www-data
cat /var/www/flaskapp/flaskapp.py
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World from flaskapp1!'
if __name__ == '__main__':
app.run(host='0.0.0.0')
cat /var/www/flaskapp2/flaskapp2.py
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World from flaskapp2!'
if __name__ == '__main__':
app.run(host='0.0.0.0')
cat /var/www/mydomain.co.uk/index.html
<!DOCTYPE html>
<html>
<body>
<h1>mydomain.co.uk</h1>
<p>This is the index page of my domain.co.uk</p>
</body>
</html>
Both virtual environments have flask installed and will run the flask apps using the development server.
I hope it's something simple that I've missed.
Looking at the uwsgi documentation for NGINX here.
Specifically:
Unfortunately nginx is not able to rewrite PATH_INFO accordingly to
SCRIPT_NAME. For such reason you need to instruct uWSGI to map
specific apps in the so called “mountpoint” and rewrite SCRIPT_NAME
and PATH_INFO automatically:
Changing my flaskapp.ini and flaskapp2.ini files to contain the mount points for the apps and turning on the manage-script-name variable has worked.
cat /etc/uwsgi/apps-available/flaskapp.ini
[uwsgi]
vhost = true
socket = /tmp/flaskapp.sock
venv = /var/www/flaskapp/venv
chdir = /var/www/flaskapp
module = flaskapp
callable = app
mount = /flaskapp=flaskapp.py
manage-script-name = true
cat /etc/uwsgi/apps-available/flaskapp2.ini
[uwsgi]
vhost = true
socket = /tmp/flaskapp2.sock
venv = /var/www/flaskapp2/venv
chdir = /var/www/flaskapp2
module = flaskapp2
callable = app
mount = /flaskapp2=flaskapp2.py
manage-script-name = true
And now both flask apps are running via uwsgi through nginx as required.
The problem is here: both flask apps have no idea that they were served from subdirectory in your domain. Nginx is passing full URL to them, not URL relative to that directory, so every URL is dispatched relative to root. Let's see that in example:
Assume that you have 3 views in your flaskapp, served on urls /one/, /two/ and /three/. So when you're trying to access view on URL /one/, you are typing address: http://yourdomain.com/flaskapp/one/. Flask will receive from nginx URL /flaskapp/one/ but there is no such view in that app, so it will send 404 in response.
What you can do is inform each flaskapp that they aren't served on domain root, but on particular subdirectory. You can achieve this by sending SCRIPT_NAME uwsgi_param with subdirectory location in value, so your nginx config will have:
location /flaskapp {
include uwsgi_params;
uwsgi_pass unix:/tmp/flaskapp.sock;
uwsgi_param SCRIPT_NAME /flaskapp;
}
location /flaskapp2 {
include uwsgi_params;
uwsgi_pass unix:/tmp/flaskapp2.sock;
uwsgi_param SCRIPT_NAME /flaskapp2;
}
Adding the mounting point worked for me.
/etc/uwsgi/apps-available/flaskapp.ini
callable = app
mount = /flaskapp=/var/www/flaskapp
/etc/uwsgi/apps-available/flaskapp2.ini
callable = app
mount = /flaskapp2=/var/www/flaskapp2
Source https://neginfinity.bitbucket.io/2017/09/hosting-multiple-flask-applications-in-nginx.html
What I have:
I have a Pyramid application that is built from a Paste ini, served by uWSGI and proxied by nginx. It works great. Here is the nginx config:
server {
listen 80;
server_name localhost;
access_log /var/log/myapp/nginx.access.log;
error_log /var/log/myapp/nginx.error.log warn;
location / {
uwsgi_pass localhost:8080;
include uwsgi_params;
}
}
Here is the uWSGI ini configuration:
[uwsgi]
socket = 127.0.0.1:8080
virtualenv = /srv/myapp/venv
die-on-term = 1
master = 1
logto = /var/log/myapp/uwsgi.log
This configuration is located inside Pyramid's production.ini, such that I serve the application with this command:
uwsgi --ini-paste-logged production.ini
All of this works just fine.
What I want to do:
One simple change. I want to serve this application as a subfolder, rather than as the root. Rather than serving it from http://localhost, I want to serve it from http://localhost/myapp.
And now everything is broken.
If I change the nginx location directive from / to /myapp or /myapp/, I get 404s, because the WSGI application receives uris that are all prepended with /myapp.
The uWSGI solution appears to be to mount the WSGI callable on the subfolder, and then pass the --manage-script-name option, at which point uWSGI should magically strip the subfolder prefix from the uri and fix the issue.
However, the documentation and every other resource I've found have only given examples of the form:
mount = /myapp=myapp.py
I don't have a myapp.py that contains a WSGI callable, because my callable is being built by PasteDeploy.
So, is it possible to mount the WSGI callable from within the Paste ini? Or am I going to have to split the uwsgi configuration out of the Paste ini and also define a separate wsgi.py with a call to paste.deploy.loadapp to generate a wsgi callable that I can mount?
Or is there another way to serve this app as a subfolder from nginx while not messing up the url reversing?
Yes, it's definitely possible to mount your Pyramid as a subdirectory with Nginx. What you'll need to use is the Modifier1 option from uWSGI like so:
location /myapp {
include uwsgi_params;
uwsgi_param SCRIPT_NAME /myapp;
uwsgi_modifier1 30;
uwsgi_pass localhost:8080;
}
The magic value of 30 tells uWSGI to remove the parameter of SCRIPT_NAME from the start of PATH_INFO in the request. Pyramid receives the request and processes it correctly.
As long as you're using the standard Pyramid machinery to generate URLs or paths within your application, SCRIPT_NAME will automatically be incorporated, meaning all URLs for links/resources etc are correct.
The documentation isn't the clearest, but there's more on the modifiers available at: https://uwsgi-docs.readthedocs.org/en/latest/Protocol.html
I wanted to do what you suggest but this is the closest solution I could find: if you are willing to modify your PasteDeploy configuration, you can follow the steps at: http://docs.pylonsproject.org/docs/pyramid/en/1.0-branch/narr/vhosting.html
Rename [app:main] to [app:mypyramidapp] and add a section reading:
[composite:main]
use = egg:Paste#urlmap
/myapp = mypyramidapp
I also had to add this to my nginx configuration:
uwsgi_param SCRIPT_NAME '';
and install the paste module
sudo pip3 install paste
I wonder if there is a way to "mount" a PasteDeploy as to original question asked...
I've hit this very problem with my deployment after switching from Python2 to Python3.
with Python2 I used the uwsgi_modifier1 30; trick, but it doesn't work anymore with Python3, as described here: https://github.com/unbit/uwsgi/issues/876
It is very badly documented (not at all? I know it from reading the uWSGI source code), but --mount option accepts the following syntax:
--mount=/app=config:/path/to/app.ini
Please note: with --mount you also need --manage-script-name option.
There are other problems with it: https://github.com/unbit/uwsgi/issues/2172
It's trivial to write a wrapper script around Paste-Deploy app, which is the way I deploy now:
from paste.script.util.logging_config import fileConfig as configure_logging
from paste.deploy import loadapp as load_app
from os import environ
config_file = environ['INI_FILE']
configure_logging(config_file)
application = load_app('config:' + config_file)
Save it to e.g. app.py and you can use it with --mount /app=app.py, the INI_FILE environment var should point to your .ini file.
As a side note - I consider moving away from uWSGI, it's buggy and documentation lacks a lot.