csrf_tokens do not match in flask & nginx - nginx

Now I am trying to develop server using nginx + unicorn + flask.
If I execute python only, csrf_tokens work without any problems.
However, if I execute python using nginx + unicorn + flask, the error occurs.
400 BAD Request - The CSRF session token is missing. or The CSRF tokens do not match.
Is there additional settings that I should have done for nginx for session?
or Did I miss something??
app/init.py
from flask_wtf.csrf import CsrfProtect
csrf = CsrfProtect()
def create_app(config_name):
app = Flask(__name__, instance_path='/instance')
app.config.from_object(config[config_name])
config[config_name].init_app(app)
bootstrap.init_app(app)
moment.init_app(app)
csrf.init_app(app)
app.config.update(CSRF_ENABLED = app.config['CSRF_ENABLED'])
//CSRF_ENABLED = True
return app
login.html
<form action="{{url_for('.login')}}" class="form-signin text-center" method="POST">
{{ form.csrf_token }}
</form>


I often use in my projects the combination of flask + gunicorn + nginx. For my forms I use a different approach:
form.py:
from wtforms import StringField, SubmitField, IntegerField
from wtforms.validators import DataRequired, Optional
class IdentityForm(FlaskForm):
age = IntegerField("Type your age", validators=[Optional()])
name = StringField("Type your name*", validators=[DataRequired()])
submit = SubmitField("Submit")
page.html
<form action="" method="post" novalidate>
{{ form.hidden_tag() }}
{{ form.age.label }}<br>
{{ form.age() }}
{{ form.name.label }}<br>
{{ form.name() }}
{{ form.submit() }}
</form>
In this small example the part that concerns us here is the argument form.hidden_tag() on the HTML's side. This argument generates a hidden field that includes a token that is used to protect the form against CSRF attacks.
For this to work it is necessary to define the variable SECRET_KEY in the flask configurations:
SECRET_KEY = os.environ.get('SECRET_KEY') or 'do-not-get-tired-youll-never-find'
The SECRET_KEY is a cryptographic key that makes it possible to generate signatures or tokens. FLASK_WTF use it to protect forms against CSRF attacks.
And that's all. FLASK_WTF takes care of the rest for you
To learn more take a look at this

Related

Providing endpoint URI to server-side rendered app

This code initializes an Apollo client for interaction with a GraphQL backend:
function create(initialState?: any) {
const isBrowser = typeof window !== "undefined"
return new ApolloClient({
connectToDevTools: isBrowser,
ssrMode: !isBrowser,
link: new HttpLink({
uri: "https://APPNAME.herokuapp.com/v1/graphql",
credentials: "same-origin",
fetch: !isBrowser && fetch
}),
cache: new InMemoryCache().restore(initialState || {})
})
}
where APPNAME is replaced with the name of the Heroku app that hosts said backend. However, when the URI written as a simple string is replaced by an environment variable such as process.env.GRAPHQL_ENDPOINT, requests are not sent to the correct URI anymore but to the /graphql endpoint on the local server. That is most certainly caused by the app being a server-side rendered Next.js app, since the request has then chances of being sent by the client rather than the server, and process doesn't exist on client-side.
Therefore the question is: what is a simple way for the app to get the endpoint URI from something like an environment variable or a file hosted on server-side?
I did some research and a solution could be hosting a microservice locally in order to host a JSON file that would contain the URI, as recommended here. Would there be a solution that doesn't require such overkill?

Made it work by adding a hidden HTML tag inside the <head> like this:
<p style={hidden} id="graphql_endpoint">
{process.env.GRAPHQL_ENDPOINT}
</p>
with hidden defined to { display: "none" }, and then retrieving it when creating the Apollo client like this:
uri: isBrowser
? document.getElementById("graphql_endpoint").innerHTML
: process.env.GRAPHQL_ENDPOINT,

Flask/NGINX 404s on only some POST requests

I have a Flask app served by Gunicorn using NGINX as a webserver. I only have two endpoints that have POST requests, a login and a submit post. The login works flawlessly however whenever I attempt to POST using the submit post endpoint, I receive a 404. My code works running on localhost without NGINX or Gunicorn and in order to detect that the request is POST vs GET I am using WTForms validate_on_submit method. The NGINX error and access logs don't show anything out of the ordinary.
Edit: Here is the code for the endpoints with post requests
New Post:
#app.route('/newpost', methods=['GET', 'POST'])
#login_required
def new_post():
form = BlogForm()
if form.validate_on_submit():
#Omitted code for getting form data and creating form object
try:
#Omitted code for database actions
return redirect(url_for('index', _external=True))
except IntegrityError:
flash("Error inserting into database")
return('', 203)
else:
#Omitted some code for template rendering
return render_template('new_post.html')
Login:
#app.route('/login', methods=['GET', 'POST'])
def login():
form = LoginForm()
userAuth = current_user.is_authenticated
if form.validate_on_submit():
flash("Login attempt logged")
user = User.query.filter_by(username=form.userID.data).first()
if user:
checkPass = form.password.data.encode('utf-8')
if bcrypt.checkpw(checkPass, user.password):
user.authenticated = True
login_user(user, remember=form.remember_me.data)
return redirect(url_for('index', _external=True))
else:
form.errors['password'] = 'false'
else:
form.errors['username'] = 'false'
else:
return render_template('login.html',
title='Log In',
loggedIn=userAuth,
form=form)

The problem ended up being with NGINX after all, I fixed it by changing the ownership of /var/lib/nginx from the nginx user to the user running the Flask application. Prior to doing this AWS wasn't letting me upload files from my form in new_post.

Task not setting variables properly?

Having some issues getting a variable passed to the client. The task that gets run is this:
def getmaps
$mapnames_current = []
url = 'http://s3-ap-northeast-1.amazonaws.com/splatoon-data.nintendo.net/stages_info.json'
resp = Net::HTTP.get_response(URI.parse(url))
buffer = resp.body
result = JSON.parse(buffer)
result.each do |gamemode|
gamemode['stages'].each do |stage|
$mapnames_current << $mapnames.key(stage['id'])
end
end
end
and $mapnames_current gets called here:
<div class="text-center">
<h2>Current Maps!</h2>
<h3>Turf War:</h3>
<h2>{{ $mapnames_current[0] }}</h2>
<h2>{{ $mapnames_current[1] }}</h2>
<h3>Ranked:</h3>
<h2>{{ $mapnames_current[2] }}</h2>
<h2>{{ $mapnames_current[3] }}</h2>
</div>
I'm not sure what's going wrong here. Shouldn't the $mapnames_current variable be accessible everywhere?

Global variables aren't synced between client and server. The only thing passed between the client and server from a task is the return values of a task. (Which should be something that can be serialized to json (or date types in json)).
There's a good screencast on tasks here: http://datamelon.io/blog/2015/creating-volt-task-objects.html

Symfony2 & phpunit: enable method override

I just started on a Symfony2 project. The CRUD generation tool created a default controller and functional test, which I'm modifying to suit my needs. The edit-form generated by the controller creates the following HTML:
<form action="/app_dev.php/invoice/7" method="post" >
<input type="hidden" name="_method" value="PUT" />
<!-- ... -->
</form>
I like the approach of overriding the HTTP method, because it enables me to create semantic routes in my application. Now I'm trying to test this form with a functional test, using the following:
$crawler = $client->click($crawler->selectLink('Edit')->link());
$form = $crawler->selectButton('Edit')->form(array(
'_method' => 'PUT',
// ...
));
$client->submit($form);
$this->assertEquals(302, $client->getResponse()->getStatusCode(), "Unexpected HTTP status code for POST /invoice/<id>/edit");
When I execute the tests by running phpunit -c /app, my tests fails because the status code is 405 instead of the expected 302.
With a bit of debugging I found out that the response of is a MethodNotAllowedHttpException. Appearantly, when running the test through PHPUnit, the method overriding (which internally maps a POST request in combination with _method=PUT param to a PUT request) doesn't take place.
That said, my question is: when executing my PHPUnit tests, why doesn't symfony recongize the overwritten method?

The second argument of form method is a http method. So try this:
$form = $crawler->selectButton('Edit')->form(array(
// ...
), 'PUT');

HTTP test server accepting GET/POST requests

I need a live test server that accepts my requests for basic information via HTTP GET and also allows me to POST (even if it's really not doing anything). This is entirely for test purposes.
A good example is here. It easily accepts GET requests, but I need one that accepts POST requests as well.
Does anyone know of a server that I can send dummy test messages too?

https://httpbin.org/
It echoes the data used in your request for any of these types:
https://httpbin.org/anything Returns most of the below.
https://httpbin.org/ip Returns Origin IP.
https://httpbin.org/user-agent Returns user-agent.
https://httpbin.org/headers Returns header dict.
https://httpbin.org/get Returns GET data.
https://httpbin.org/post Returns POST data.
https://httpbin.org/put Returns PUT data.
https://httpbin.org/delete Returns DELETE data
https://httpbin.org/gzip Returns gzip-encoded data.
https://httpbin.org/status/:code Returns given HTTP Status code.
https://httpbin.org/response-headers?key=val Returns given response headers.
https://httpbin.org/redirect/:n 302 Redirects n times.
https://httpbin.org/relative-redirect/:n 302 Relative redirects n times.
https://httpbin.org/cookies Returns cookie data.
https://httpbin.org/cookies/set/:name/:value Sets a simple cookie.
https://httpbin.org/basic-auth/:user/:passwd Challenges HTTPBasic Auth.
https://httpbin.org/hidden-basic-auth/:user/:passwd 404'd BasicAuth.
https://httpbin.org/digest-auth/:qop/:user/:passwd Challenges HTTP Digest Auth.
https://httpbin.org/stream/:n Streams n–100 lines.
https://httpbin.org/delay/:n Delays responding for n–10 seconds.

There is http://ptsv2.com/
"Here you will find a server which receives any POST you wish to give it and stores the contents for you to review."

Webhook Tester is a great tool: https://webhook.site (GitHub)
Important for me, it showed the IP of the requester, which is helpful when you need to whitelist an IP address but aren't sure what it is.

nc one-liner local test server
Setup a local test server in one line under Linux:
nc -kdl localhost 8000
Sample request maker on another shell:
wget http://localhost:8000
then on the first shell you see the request showed up:
GET / HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: localhost:8000
Connection: Keep-Alive
nc from the netcat-openbsd package is widely available and pre-installed on Ubuntu.
Tested on Ubuntu 18.04.

http://requestb.in was similar to the already mentioned tools and also had a very nice UI.
RequestBin gives you a URL that will collect requests made to it and let you inspect them in a human-friendly way.
Use RequestBin to see what your HTTP client is sending or to inspect and debug webhook requests.
Though it has been discontinued as of Mar 21, 2018.
We have discontinued the publicly hosted version of RequestBin due to ongoing abuse that made it very difficult to keep the site up reliably. Please see instructions for setting up your own self-hosted instance.

If you want a local test server that accepts any URL and just dumps the request to the console, you can use node:
const http = require("http");
const hostname = "0.0.0.0";
const port = 3000;
const server = http.createServer((req, res) => {
console.log(`\n${req.method} ${req.url}`);
console.log(req.headers);
req.on("data", function(chunk) {
console.log("BODY: " + chunk);
});
res.statusCode = 200;
res.setHeader("Content-Type", "text/plain");
res.end("Hello World\n");
});
server.listen(port, hostname, () => {
console.log(`Server running at http://localhost:${port}/`);
});
Save it in a file 'echo.js' and run it as follows:
$ node echo.js
Server running at http://localhost:3000/
You can then submit data:
$ curl -d "[1,2,3]" -XPOST http://localhost:3000/foo/bar
which will be shown in the server's stdout:
POST /foo/bar
{ host: 'localhost:3000',
'user-agent': 'curl/7.54.1',
accept: '*/*',
'content-length': '7',
'content-type': 'application/x-www-form-urlencoded' }
BODY: [1,2,3]

Have a look at PutsReq, it's similar to the others, but it also allows you to write the responses you want using JavaScript.

Here is one Postman echo: https://docs.postman-echo.com/
example:
curl --request POST \
--url https://postman-echo.com/post \
--data 'This is expected to be sent back as part of response body.'
response:
{"args":{},"data":"","files":{},"form":{"This is expected to be sent back as part of response body.":""},"headers":{"host":"postman-echo.com","content-length":"58","accept":"*/*","content-type":"application/x-www-form-urlencoded","user-agent":"curl/7.54.0","x-forwarded-port":"443","x-forwarded-proto":"https"},"json":{"...

You can run the actual Ken Reitz's httpbin server locally (under docker or on bare metal):
https://github.com/postmanlabs/httpbin
Run dockerized
docker pull kennethreitz/httpbin
docker run -p 80:80 kennethreitz/httpbin
Run directly on your machine
## install dependencies
pip3 install gunicorn decorator httpbin werkzeug Flask flasgger brotlipy gevent meinheld six pyyaml
## start the server
gunicorn -b 0.0.0.0:8000 httpbin:app -k gevent
Now you have your personal httpbin instance running on http://0.0.0.0:8000 (visible to all of your LAN)
Minimal Flask REST server
I wanted a server which returns predefined responses so I found that in this case it's simpler to use a minimal Flask app:
#!/usr/bin/env python3
# Install dependencies:
# pip3 install flask
import json
from flask import Flask, request, jsonify
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST'])
def root():
# spit back whatever was posted + the full env
return jsonify(
{
'request.json': request.json,
'request.values': request.values,
'env': json.loads(json.dumps(request.__dict__, sort_keys=True, default=str))
}
)
#app.route('/post', methods=['GET', 'POST'])
def post():
if not request.json:
return 'No JSON payload! Expecting POST!'
# return the literal POST-ed payload
return jsonify(
{
'payload': request.json,
}
)
#app.route('/users/<gid>', methods=['GET', 'POST'])
def users(gid):
# return a JSON list of users in a group
return jsonify([{'user_id': i,'group_id': gid } for i in range(42)])
#app.route('/healthcheck', methods=['GET'])
def healthcheck():
# return some JSON
return jsonify({'key': 'healthcheck', 'status': 200})
if __name__ == "__main__":
with app.test_request_context():
app.debug = True
app.run(debug=True, host='0.0.0.0', port=8000)

I don't konw why all of the answers here make a very simple work very hard!
When there is a request on HTTP, actually a client will send a HTTP_MESSAGE to server (read about what is HTTP_MESSAGE) and you can make a server in just 2 simple steps:
Install netcat:
In many unix-based systems you have this already installed and if you have windows just google it , the installation process is really simple, you just need a nc.exe file and then you should copy the path of this nc.exe file to your path environment variable and check if every thing is OK with nc -h
Make a server which is listening on localhost:12345:
just type nc -l -p 12345 on your terminal and everything is done! (in mac nc -l 12345 tnx Silvio Biasiol)
Now you have a server which is listening on http://localhost:12345 , for example you can make a post request with axios If you are a js developer:
axios.post('http://localhost:12345', { firstName: 'Fred' })
or make your own xhr or make a form in a HTML file and submit it to server, sth. like:
<form action="http://localhost:12345" method="post">
or make a request with curl or wget or etc. Then check your terminal, a raw HTTP_MESSAGE should be appear on your terminal and you can start your happy hacking ;)

https://www.mockable.io. It has nice feature of getting endpoints without login (24h temporary account)

Create choose a free web host and put the following code
<h1>Request Headers</h1>
<?php
$headers = apache_request_headers();
foreach ($headers as $header => $value) {
echo "<b>$header:</b> $value <br />\n";
}
?>

I have created an open-source hackable local testing server that you can get running in minutes. You can create new API's, define your own response and hack it in any ways you wish to.
Github Link : https://github.com/prabodhprakash/localTestingServer

some online httpbin:
https://httpbin.org/
https://httpbingo.org/
https://quic.aiortc.org/httpbin/
get client ip, port, ua..
http://ifconfig.io/
get client ip, isp
https://www.cip.cc/

If you need or want a simple HTTP server with the following:
Can be run locally or in a network sealed from the public Internet
Has some basic auth
Handles POST requests
I built one on top of the excellent SimpleHTTPAuthServer already on PyPI. This adds handling of POST requests:
https://github.com/arielampol/SimpleHTTPAuthServerWithPOST
Otherwise, all the other options publicly available are already so good and robust.

You might don't need any web site for that, only open up the browser, press F12 to get access to developer tools > console, then in console write some JavaScript Code to do that.
Here I share some ways to accomplish that:
For GET request:
*.Using jQuery:
$.get("http://someurl/status/?messageid=597574445", function(data, status){
console.log(data, status);
});
For POST request:
Using jQuery $.ajax:
var url= "http://someurl/",
api_key = "6136-bc16-49fb-bacb-802358",
token1 = "Just for test",
result;
$.ajax({
url: url,
type: "POST",
data: {
api_key: api_key,
token1: token1
},
}).done(function(result) {
console.log("done successfuly", result);
}).fail(function(error) {
console.log(error.responseText, error);
});
Using jQuery, append and submit
var merchantId = "AA86E",
token = "4107120133142729",
url = "https://payment.com/Index";
var form = `<form id="send-by-post" method="post" action="${url}">
<input id="token" type="hidden" name="token" value="${merchantId}"/>
<input id="merchantId" name="merchantId" type="hidden" value="${token}"/>
<button type="submit" >Pay</button>
</div>
</form> `;
$('body').append(form);
$("#send-by-post").submit();//Or $(form).appendTo("body").submit();
Using Pure JavaScript:
`var api_key = "73736-bc16-49fb-bacb-643e58",
recipient = "095552565",
token1 = "4458",
url = 'http://smspanel.com/send/';`
``var form = `<form id="send-by-post" method="post" action="${url}">
<input id="api_key" type="hidden" name="api_key" value="${api_key}"/>
<input id="recipient" type="hidden" name="recipient" value="${recipient}"/>
<input id="token1" name="token1" type="hidden" value="${token1}"/>
<button type="submit" >Send</button>
</div>
</form>`;``
document.querySelector("body").insertAdjacentHTML('beforeend',form);
document.querySelector("#send-by-post").submit();
Or even using ASP.Net:
var url = "https://Payment.com/index";
Response.Clear();
var sb = new System.Text.StringBuilder();
sb.Append("<html>");
sb.AppendFormat("<body onload='document.forms[0].submit()'>");
sb.AppendFormat("<form action='{0}' method='post'>", url);
sb.AppendFormat("<input type='hidden' name='merchantId' value='{0}'>", "C668");
sb.AppendFormat("<input type='hidden' name='Token' value='{0}'>", "22720281459");
sb.Append("</form>");
sb.Append("</body>");
sb.Append("</html>");
Response.Write(sb.ToString());
Response.End();

I am not sure if anyone would take this much pain to test GET and POST calls. I took Python Flask module and wrote a function that does something similar to what #Robert shared.
from flask import Flask, request
app = Flask(__name__)
#app.route('/method', methods=['GET', 'POST'])
#app.route('/method/<wish>', methods=['GET', 'POST'])
def method_used(wish=None):
if request.method == 'GET':
if wish:
if wish in dir(request):
ans = None
s = "ans = str(request.%s)" % wish
exec s
return ans
else:
return 'This wish is not available. The following are the available wishes: %s' % [method for method in dir(request) if '_' not in method]
else:
return 'This is just a GET method'
else:
return "You are using POST"
When I run this, this follows:
C:\Python27\python.exe E:/Arindam/Projects/Flask_Practice/first.py
* Restarting with stat
* Debugger is active!
* Debugger PIN: 581-155-269
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Now lets try some calls. I am using the browser.
http://127.0.0.1:5000/method
This is just a GET method
http://127.0.0.1:5000/method/NotCorrect
This wish is not available. The following are the available wishes:
['application', 'args', 'authorization', 'blueprint', 'charset', 'close', 'cookies', 'data', 'date', 'endpoint', 'environ', 'files', 'form', 'headers', 'host', 'json', 'method', 'mimetype', 'module', 'path', 'pragma', 'range', 'referrer', 'scheme', 'shallow', 'stream', 'url', 'values']
http://127.0.0.1:5000/method/environ
{'wsgi.multiprocess': False, 'HTTP_COOKIE': 'csrftoken=YFKYYZl3DtqEJJBwUlap28bLG1T4Cyuq', 'SERVER_SOFTWARE': 'Werkzeug/0.12.2', 'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET', 'PATH_INFO': '/method/environ', 'SERVER_PROTOCOL': 'HTTP/1.1', 'QUERY_STRING': '', 'werkzeug.server.shutdown': , 'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36', 'HTTP_CONNECTION': 'keep-alive', 'SERVER_NAME': '127.0.0.1', 'REMOTE_PORT': 49569, 'wsgi.url_scheme': 'http', 'SERVER_PORT': '5000', 'werkzeug.request': , 'wsgi.input': , 'HTTP_HOST': '127.0.0.1:5000', 'wsgi.multithread': False, 'HTTP_UPGRADE_INSECURE_REQUESTS': '1', 'HTTP_ACCEPT': "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", 'wsgi.version': (1, 0), 'wsgi.run_once': False, 'wsgi.errors': ", mode 'w' at 0x0000000002042150>", 'REMOTE_ADDR': '127.0.0.1', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, sdch, br'}

Another one that offers some customization and is easy to use (no install, signup) is https://beeceptor.com .
You create a endpoint, makes a initial request to it and can tweak the responses.

I am using this REST API all the time: https://restful-api.dev
It stores the created objects indefinitely.
Also, the schema is quite flexible, you can pass any JSON data.
I am a Front-End developer and is very useful when I need to create some sample data. This is the only one I could find that does it for free without any registration or tokens.

Just set one up yourself. Copy this snippet to your webserver.
echo "<pre>";
print_r($_POST);
echo "</pre>";
Just post what you want to that page. Done.

Resources