Logstash and Flowdock integration - http

I'm trying to send a subset of messages to Flowdock as output from Logstash. Unfortunately, due to this issue I get essentially nothing back about why my messages aren't making it. Stripping down to a basic example, I see the problem even if I change my output config to the following:
output {
http {
http_method => "post"
url => "https://api.flowdock.com/v1/messages/team_inbox/API_TOKEN"
format => "message"
content_type => "application/json"
message => "{\"source\":\"logstash\",\"from_address\":\"me#example.com\", \"subject\":\"Log Message\", \"content\":\"test\"}"
}
}
I know, though, that output is generally working because if I add the following to output I see log messages written to the file:
file {
path => "/mnt/test.log"
}
I also know that the http message I'm sending to Flowdock should work since
curl -X POST https://api.flowdock.com/v1/messages/team_inbox/API_TOKEN -d "{\"source\":\"logstash\",\"from_address\":\"me#example.com\",\"subject\":\"Log Message\",\"content\":\"test\"}" -H "Content-Type: application/json"
results in a message being posted to the team inbox.
Are there any ways to work around this issue to determine why my output from logstash is failing?

I would start debugging the issue by first sending the requests from Logstash to a service that just outputs the received request, for example RequestBin.
Something like:
output {
http {
http_method => "post"
url => "http://requestb.in/<created_id>"
format => "message"
content_type => "application/json"
message => "{\"source\":\"logstash\",\"from_address\":\"me#example.com\",\"subject\":\"Log Message\", \"content\":\"test\"}"
}
}
After you've made sure that the request Logstash is making is correct, take that request (preferably the exact data) and try to send it to Flowdock using curl or some other means.
At this point you should be able to tell why the request fails in either end and notify the party accordingly (i.e. open a ticket to https://logstash.jira.com/secure/Dashboard.jspa or send an email to support#flowdock.com).

Related

Cypress: How to access the body of a POST-request?

Is there a way in Cypress to check the body of a POST-request?
E.g.: I have entered some data in a form, then pressed "Submit".
The form-data is send via POST-request, separated by a blank line from the header-data.
I would like to check the form-data. If all data, which I have entered, are included and if they are correct.
Is that possible with Cypress?
cy.get('#login').then(function (xhr) {
const body = xhr.requestBody;
console.log(xhr);
expect(xhr.method).to.eq('POST');
});
The xhr-object doesn't have the transferred data included.
It should be possible.
describe('Capturing data sent by the form via POST method', () => {
before(() => {
Cypress.config('baseUrl', 'https://www.reddit.com');
cy.server();
cy.route({
method: 'POST',
url: '/login'
}).as('redditLogin');
});
it('should capture the login and password', () => {
cy.visit('/login');
cy.get('#loginUsername').type('username');
cy.get('#loginPassword').type('password');
cy.get('button[type="submit"]').click();
cy.wait('#redditLogin').then(xhr => {
cy.log(xhr.responseBody);
cy.log(xhr.requestBody);
expect(xhr.method).to.eq('POST');
})
});
});
This is how you can inspect your data in Chrome Developer Tool.
You should see the same thing you've seen from Chrome Developer Tool when you run your test in Cypress.
I was Googling the same problem and somehow landed here before reaching the documentation.
Anyway, have you tried something like:
cy.wait('#login').should((xhr) => {
const body = xhr.request.body
expect(body).to.match(/email/)
})
I haven't tested it out with a multipart/form-data encoded request, but I suspect that you'll also find the request body that way.
Good luck!
It's better to use cy.intercept() in order to spy, stub and assert network requests and responses.
// assert that a request to this route
// was made with a body that included 'user'
cy.wait('#someRoute').its('request.body').should('include', 'user')
// assert that a request to this route
// received a response with HTTP status 500
cy.wait('#someRoute').its('response.statusCode').should('eq', 500)
// assert that a request to this route
// received a response body that includes 'id'
cy.wait('#someRoute').its('response.body').should('include', 'id')
Link to the docs

Flutter HTTP request with JSON data to a Firebase function

I am currently trying to set up our Firestore database for a Flutter mobile app to take data from a Firebase function and store it into the database. Currently, we are trying to pass an HTTP request to our URI and attach some JSON data to it.
My function is as follows:
exports.testFunction = functions.https.onRequest((request, response) => {
var data = {
name: request.body.name,
age: request.body.age
};
var setDoc = db.collection('users').add(data);
response.json({result: `User ${data.name} at age ${data.age} added.`});
return;
});
Currently, the data can be retrieved when using a web browser and requesting from https://us-central1-[project-name].cloudfunctions.net/testFunction?name=[string]&age=[int]. This also works within our Flutter app, using the entire URL as a single argument with no body. However, when trying a cURL request from the command line:
curl -H 'Content-Type: application/json' -d '{"name": "[string]", "age": [int]}' https://us-central1-[project-name].cloudfunctions.net/testFunction
Hypothetically, the data should be in request.body, but I am getting an error saying the request could not be handled. What exactly is going wrong here?
Later, we want to implement this request in a Dart/Flutter app by attaching the JSON object to the body of the http.post. Is there a missing link in between this process that I'm missing?
The error Request body is missing data means that the payload that's being sent in the request needs to be mapped inside data. It's mentioned here in the docs that the Request body should have a data field.
var data = {
'data':{
name: request.body.name,
age: request.body.age
}
}

Computer Vision API - v1.0 Recognize Handwritten Text returns empty response

I am trying to start using the computer vision API but I keep getting an empty response. My request in php (as exported by Postman) looks like this:
<?php
$request = new HttpRequest();
$request->setUrl('https://westcentralus.api.cognitive.microsoft.com/vision/v1.0/recognizeText');
$request->setMethod(HTTP_METH_POST);
$request->setQueryData(array(
'language' => 'en',
'handwriting' => 'true'
));
$request->setHeaders(array(
'Postman-Token' => '442d04f7-49a0-4262-9d0f-666fe5240cc7',
'Cache-Control' => 'no-cache',
'Content-Type' => 'application/octet-stream',
'Ocp-Apim-Subscription-Key' => 'KEY'
));
try {
$response = $request->send();
echo $response->getBody();
} catch (HttpException $ex) {
echo $ex;
}
The above code works fine with the ocr endpoint!
The file is passed as binary using Postman.
Edit: I also tried to copy/paste the code from here: https://learn.microsoft.com/en-gb/azure/cognitive-services/computer-vision/quickstarts/php#ocr-php-example-request and if I change the ocr endpoint to recognizeText I get an empty response as well!
Unlike the other Computer Vision endpoints, RecognizeText is an asynchronous operation. Barring some issue with the image, you will get a 202 response instead of the usual 200 response. 202 responses customarily contain an empty response body. In this particular case you can find the URL where you can query for completion of the task. The documentation is here. The header you're looking for is Operation-Location.

Put a custom http header in backbone

I am creating an API with Tastypie and I want to access to the API from Backbone.
To send credentials I use an user_id and a api_key. I do this in android and with curl and this work great, but I can set the http header from backbone.
In curl I use:
curl --dump-header - -H "Accept: application/json" -H "Content-Type: application/json" -H "user_id: 32" -H "api_key: 69950" -X DELETE "http://127.0.0.1:8000/api/v1/deletenote/66/?format=json"
and in android java I use:
HttpDelete requestDELETE = new HttpDelete();
requestDELETE.setHeader("Content-type", "application/json");
requestDELETE.setHeader("Accept", "application/json");
requestDELETE.setHeader(Constants.HEADER_USER_ID, user_id);
requestDELETE.addHeader(Constants.HEADER_API_KEY, key);
Both of them work great, but when I try this in Backbone following the responses that I found in other post from the page, this didn't work.
I am trying this:
var removeNote = new DeleteNoteModel({id:this.model.toJSON().id},{ query:this.model.toJSON().id});
removeNote.destroy({
headers: {'user_id':dataWeb.get("id"),'api_key':dataWeb.get("api_key")}
},{
async:false,
error: function(model, response){
console.log("KO_REMOVE_NOTE");
console.log(response);
},
success : function(model, response){
console.log("OK_REMOVE_NOTE");
console.log(response);
}
}
);
I'm putting the header when I call to the destroy call, but this don't send anithing to the server.
What I am doing in a wrong mode?
Thanks to all.
Tallmaris answer should fix it for you though I would recommend usign jQuery ajaxSetup method to setup the headers as default values for all ajax requests as I believe you need them all the time anyway right?
Somewhere where you launch the App put in
$.ajaxSetup({
headers: {
'user_id':dataWeb.get("id"),
'api_key':dataWeb.get("api_key")
}
});
Thanks to that you'll save yourself a lot of repeated code :) keep it DRY!
(obviously you'd need to ensure that dataWeb is available in the scope of where you launch the app :) )
It seems you are passing two parameters to destroy, pass only one containing the headers and the other options together, unless the brackets order is a typo. Try this:
removeNote.destroy({
headers: {
'user_id':dataWeb.get("id"),
'api_key':dataWeb.get("api_key")
}, // there was an extra close-open curly here...
async:false,
error: function(model, response){
console.log("KO_REMOVE_NOTE");
console.log(response);
},
success : function(model, response){
console.log("OK_REMOVE_NOTE");
console.log(response);
}
});

HTTP test server accepting GET/POST requests

I need a live test server that accepts my requests for basic information via HTTP GET and also allows me to POST (even if it's really not doing anything). This is entirely for test purposes.
A good example is here. It easily accepts GET requests, but I need one that accepts POST requests as well.
Does anyone know of a server that I can send dummy test messages too?
https://httpbin.org/
It echoes the data used in your request for any of these types:
https://httpbin.org/anything Returns most of the below.
https://httpbin.org/ip Returns Origin IP.
https://httpbin.org/user-agent Returns user-agent.
https://httpbin.org/headers Returns header dict.
https://httpbin.org/get Returns GET data.
https://httpbin.org/post Returns POST data.
https://httpbin.org/put Returns PUT data.
https://httpbin.org/delete Returns DELETE data
https://httpbin.org/gzip Returns gzip-encoded data.
https://httpbin.org/status/:code Returns given HTTP Status code.
https://httpbin.org/response-headers?key=val Returns given response headers.
https://httpbin.org/redirect/:n 302 Redirects n times.
https://httpbin.org/relative-redirect/:n 302 Relative redirects n times.
https://httpbin.org/cookies Returns cookie data.
https://httpbin.org/cookies/set/:name/:value Sets a simple cookie.
https://httpbin.org/basic-auth/:user/:passwd Challenges HTTPBasic Auth.
https://httpbin.org/hidden-basic-auth/:user/:passwd 404'd BasicAuth.
https://httpbin.org/digest-auth/:qop/:user/:passwd Challenges HTTP Digest Auth.
https://httpbin.org/stream/:n Streams n–100 lines.
https://httpbin.org/delay/:n Delays responding for n–10 seconds.
There is http://ptsv2.com/
"Here you will find a server which receives any POST you wish to give it and stores the contents for you to review."
Webhook Tester is a great tool: https://webhook.site (GitHub)
Important for me, it showed the IP of the requester, which is helpful when you need to whitelist an IP address but aren't sure what it is.
nc one-liner local test server
Setup a local test server in one line under Linux:
nc -kdl localhost 8000
Sample request maker on another shell:
wget http://localhost:8000
then on the first shell you see the request showed up:
GET / HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: localhost:8000
Connection: Keep-Alive
nc from the netcat-openbsd package is widely available and pre-installed on Ubuntu.
Tested on Ubuntu 18.04.
http://requestb.in was similar to the already mentioned tools and also had a very nice UI.
RequestBin gives you a URL that will collect requests made to it and let you inspect them in a human-friendly way.
Use RequestBin to see what your HTTP client is sending or to inspect and debug webhook requests.
Though it has been discontinued as of Mar 21, 2018.
We have discontinued the publicly hosted version of RequestBin due to ongoing abuse that made it very difficult to keep the site up reliably. Please see instructions for setting up your own self-hosted instance.
If you want a local test server that accepts any URL and just dumps the request to the console, you can use node:
const http = require("http");
const hostname = "0.0.0.0";
const port = 3000;
const server = http.createServer((req, res) => {
console.log(`\n${req.method} ${req.url}`);
console.log(req.headers);
req.on("data", function(chunk) {
console.log("BODY: " + chunk);
});
res.statusCode = 200;
res.setHeader("Content-Type", "text/plain");
res.end("Hello World\n");
});
server.listen(port, hostname, () => {
console.log(`Server running at http://localhost:${port}/`);
});
Save it in a file 'echo.js' and run it as follows:
$ node echo.js
Server running at http://localhost:3000/
You can then submit data:
$ curl -d "[1,2,3]" -XPOST http://localhost:3000/foo/bar
which will be shown in the server's stdout:
POST /foo/bar
{ host: 'localhost:3000',
'user-agent': 'curl/7.54.1',
accept: '*/*',
'content-length': '7',
'content-type': 'application/x-www-form-urlencoded' }
BODY: [1,2,3]
Have a look at PutsReq, it's similar to the others, but it also allows you to write the responses you want using JavaScript.
Here is one Postman echo: https://docs.postman-echo.com/
example:
curl --request POST \
--url https://postman-echo.com/post \
--data 'This is expected to be sent back as part of response body.'
response:
{"args":{},"data":"","files":{},"form":{"This is expected to be sent back as part of response body.":""},"headers":{"host":"postman-echo.com","content-length":"58","accept":"*/*","content-type":"application/x-www-form-urlencoded","user-agent":"curl/7.54.0","x-forwarded-port":"443","x-forwarded-proto":"https"},"json":{"...
You can run the actual Ken Reitz's httpbin server locally (under docker or on bare metal):
https://github.com/postmanlabs/httpbin
Run dockerized
docker pull kennethreitz/httpbin
docker run -p 80:80 kennethreitz/httpbin
Run directly on your machine
## install dependencies
pip3 install gunicorn decorator httpbin werkzeug Flask flasgger brotlipy gevent meinheld six pyyaml
## start the server
gunicorn -b 0.0.0.0:8000 httpbin:app -k gevent
Now you have your personal httpbin instance running on http://0.0.0.0:8000 (visible to all of your LAN)
Minimal Flask REST server
I wanted a server which returns predefined responses so I found that in this case it's simpler to use a minimal Flask app:
#!/usr/bin/env python3
# Install dependencies:
# pip3 install flask
import json
from flask import Flask, request, jsonify
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST'])
def root():
# spit back whatever was posted + the full env
return jsonify(
{
'request.json': request.json,
'request.values': request.values,
'env': json.loads(json.dumps(request.__dict__, sort_keys=True, default=str))
}
)
#app.route('/post', methods=['GET', 'POST'])
def post():
if not request.json:
return 'No JSON payload! Expecting POST!'
# return the literal POST-ed payload
return jsonify(
{
'payload': request.json,
}
)
#app.route('/users/<gid>', methods=['GET', 'POST'])
def users(gid):
# return a JSON list of users in a group
return jsonify([{'user_id': i,'group_id': gid } for i in range(42)])
#app.route('/healthcheck', methods=['GET'])
def healthcheck():
# return some JSON
return jsonify({'key': 'healthcheck', 'status': 200})
if __name__ == "__main__":
with app.test_request_context():
app.debug = True
app.run(debug=True, host='0.0.0.0', port=8000)
I don't konw why all of the answers here make a very simple work very hard!
When there is a request on HTTP, actually a client will send a HTTP_MESSAGE to server (read about what is HTTP_MESSAGE) and you can make a server in just 2 simple steps:
Install netcat:
In many unix-based systems you have this already installed and if you have windows just google it , the installation process is really simple, you just need a nc.exe file and then you should copy the path of this nc.exe file to your path environment variable and check if every thing is OK with nc -h
Make a server which is listening on localhost:12345:
just type nc -l -p 12345 on your terminal and everything is done! (in mac nc -l 12345 tnx Silvio Biasiol)
Now you have a server which is listening on http://localhost:12345 , for example you can make a post request with axios If you are a js developer:
axios.post('http://localhost:12345', { firstName: 'Fred' })
or make your own xhr or make a form in a HTML file and submit it to server, sth. like:
<form action="http://localhost:12345" method="post">
or make a request with curl or wget or etc. Then check your terminal, a raw HTTP_MESSAGE should be appear on your terminal and you can start your happy hacking ;)
https://www.mockable.io. It has nice feature of getting endpoints without login (24h temporary account)
Create choose a free web host and put the following code
<h1>Request Headers</h1>
<?php
$headers = apache_request_headers();
foreach ($headers as $header => $value) {
echo "<b>$header:</b> $value <br />\n";
}
?>
I have created an open-source hackable local testing server that you can get running in minutes. You can create new API's, define your own response and hack it in any ways you wish to.
Github Link : https://github.com/prabodhprakash/localTestingServer
some online httpbin:
https://httpbin.org/
https://httpbingo.org/
https://quic.aiortc.org/httpbin/
get client ip, port, ua..
http://ifconfig.io/
get client ip, isp
https://www.cip.cc/
If you need or want a simple HTTP server with the following:
Can be run locally or in a network sealed from the public Internet
Has some basic auth
Handles POST requests
I built one on top of the excellent SimpleHTTPAuthServer already on PyPI. This adds handling of POST requests:
https://github.com/arielampol/SimpleHTTPAuthServerWithPOST
Otherwise, all the other options publicly available are already so good and robust.
You might don't need any web site for that, only open up the browser, press F12 to get access to developer tools > console, then in console write some JavaScript Code to do that.
Here I share some ways to accomplish that:
For GET request:
*.Using jQuery:
$.get("http://someurl/status/?messageid=597574445", function(data, status){
console.log(data, status);
});
For POST request:
Using jQuery $.ajax:
var url= "http://someurl/",
api_key = "6136-bc16-49fb-bacb-802358",
token1 = "Just for test",
result;
$.ajax({
url: url,
type: "POST",
data: {
api_key: api_key,
token1: token1
},
}).done(function(result) {
console.log("done successfuly", result);
}).fail(function(error) {
console.log(error.responseText, error);
});
Using jQuery, append and submit
var merchantId = "AA86E",
token = "4107120133142729",
url = "https://payment.com/Index";
var form = `<form id="send-by-post" method="post" action="${url}">
<input id="token" type="hidden" name="token" value="${merchantId}"/>
<input id="merchantId" name="merchantId" type="hidden" value="${token}"/>
<button type="submit" >Pay</button>
</div>
</form> `;
$('body').append(form);
$("#send-by-post").submit();//Or $(form).appendTo("body").submit();
Using Pure JavaScript:
`var api_key = "73736-bc16-49fb-bacb-643e58",
recipient = "095552565",
token1 = "4458",
url = 'http://smspanel.com/send/';`
``var form = `<form id="send-by-post" method="post" action="${url}">
<input id="api_key" type="hidden" name="api_key" value="${api_key}"/>
<input id="recipient" type="hidden" name="recipient" value="${recipient}"/>
<input id="token1" name="token1" type="hidden" value="${token1}"/>
<button type="submit" >Send</button>
</div>
</form>`;``
document.querySelector("body").insertAdjacentHTML('beforeend',form);
document.querySelector("#send-by-post").submit();
Or even using ASP.Net:
var url = "https://Payment.com/index";
Response.Clear();
var sb = new System.Text.StringBuilder();
sb.Append("<html>");
sb.AppendFormat("<body onload='document.forms[0].submit()'>");
sb.AppendFormat("<form action='{0}' method='post'>", url);
sb.AppendFormat("<input type='hidden' name='merchantId' value='{0}'>", "C668");
sb.AppendFormat("<input type='hidden' name='Token' value='{0}'>", "22720281459");
sb.Append("</form>");
sb.Append("</body>");
sb.Append("</html>");
Response.Write(sb.ToString());
Response.End();
I am not sure if anyone would take this much pain to test GET and POST calls. I took Python Flask module and wrote a function that does something similar to what #Robert shared.
from flask import Flask, request
app = Flask(__name__)
#app.route('/method', methods=['GET', 'POST'])
#app.route('/method/<wish>', methods=['GET', 'POST'])
def method_used(wish=None):
if request.method == 'GET':
if wish:
if wish in dir(request):
ans = None
s = "ans = str(request.%s)" % wish
exec s
return ans
else:
return 'This wish is not available. The following are the available wishes: %s' % [method for method in dir(request) if '_' not in method]
else:
return 'This is just a GET method'
else:
return "You are using POST"
When I run this, this follows:
C:\Python27\python.exe E:/Arindam/Projects/Flask_Practice/first.py
* Restarting with stat
* Debugger is active!
* Debugger PIN: 581-155-269
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Now lets try some calls. I am using the browser.
http://127.0.0.1:5000/method
This is just a GET method
http://127.0.0.1:5000/method/NotCorrect
This wish is not available. The following are the available wishes:
['application', 'args', 'authorization', 'blueprint', 'charset', 'close', 'cookies', 'data', 'date', 'endpoint', 'environ', 'files', 'form', 'headers', 'host', 'json', 'method', 'mimetype', 'module', 'path', 'pragma', 'range', 'referrer', 'scheme', 'shallow', 'stream', 'url', 'values']
http://127.0.0.1:5000/method/environ
{'wsgi.multiprocess': False, 'HTTP_COOKIE': 'csrftoken=YFKYYZl3DtqEJJBwUlap28bLG1T4Cyuq', 'SERVER_SOFTWARE': 'Werkzeug/0.12.2', 'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET', 'PATH_INFO': '/method/environ', 'SERVER_PROTOCOL': 'HTTP/1.1', 'QUERY_STRING': '', 'werkzeug.server.shutdown': , 'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36', 'HTTP_CONNECTION': 'keep-alive', 'SERVER_NAME': '127.0.0.1', 'REMOTE_PORT': 49569, 'wsgi.url_scheme': 'http', 'SERVER_PORT': '5000', 'werkzeug.request': , 'wsgi.input': , 'HTTP_HOST': '127.0.0.1:5000', 'wsgi.multithread': False, 'HTTP_UPGRADE_INSECURE_REQUESTS': '1', 'HTTP_ACCEPT': "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", 'wsgi.version': (1, 0), 'wsgi.run_once': False, 'wsgi.errors': ", mode 'w' at 0x0000000002042150>", 'REMOTE_ADDR': '127.0.0.1', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, sdch, br'}
Another one that offers some customization and is easy to use (no install, signup) is https://beeceptor.com .
You create a endpoint, makes a initial request to it and can tweak the responses.
I am using this REST API all the time: https://restful-api.dev
It stores the created objects indefinitely.
Also, the schema is quite flexible, you can pass any JSON data.
I am a Front-End developer and is very useful when I need to create some sample data. This is the only one I could find that does it for free without any registration or tokens.
Just set one up yourself. Copy this snippet to your webserver.
echo "<pre>";
print_r($_POST);
echo "</pre>";
Just post what you want to that page. Done.

Resources