I am trying to POST some raw data in a functional RSpec example:
payload = "{wrongly_formatted_json}"
uri = "http://example.com"
headers["HTTP_CONTENT_TYPE"] = "application/json"
post(uri, payload, headers)
RSpec is too smart and wraps my request payload in another hash:
{"{wrongly_formatted_json}" => nil}
I found people recommending #request.env['RAW_POST_DATA'] but the request object is not available in my example. Am I right that the #request object comes with ActiveSupport::TestCase and as such can not be used or is it because I include the Rack::Test::Methods?
How can I post raw data with RSpec? Note, I want to send wrongly formatted JSON. I am using rspec-rails 2.12.0.
Here is a similar SO question asking the same question but they use ActiveSupport::TestCase to solve it.
Related
I am creating a server using Go that allows the client to upload a file and then use a server function to parse the file. Currently, I am using two separate requests:
1) First request sends the file the user has uploaded
2) Second request sends the parameters to the server that the server needs to parse the file.
However, I have realised that due to the nature of the program, there can be concurrency problem if multiple users try to use the server at the same time. My solution to that was using mutex locks. However, I am receiving the file, sending a response, and then receiving the parameters and it seems that Go cannot send a response back when the mutex is locked. I am thinking about solving this problem by sending both the file and the parameters in one single HTTP request. Is there a way to do that? Thanks
Sample code (only relevant parts):
Code to send file from client:
handleUpload() {
const data = new FormData()
for(var x = 0; x < this.state.selectedFile.length; x++) {
data.append('myFile', this.state.selectedFile[x])
}
var self = this;
let url = *the appropriate url*
axios.post(url, data, {})
.then(res => {
//other logic
self.handleParser();
})
}
Code for handleParser():
handleNessusParser(){
let parserParameter = {
SourcePath : location,
ProjectName : this.state.projectName
}
// fetch the the response from the server
let self = this;
let url = *url*
fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(parserParameter),
}).then( (response) => {
if(response.status === 200) {
//success logic
}
}).catch (function (error) {
console.log("error: ", error);
});
}
The question is not really about Go or reactjs or any particular software library.
To solve your problem you'd first need to understand how HTTP POST works,
hence I invite you to first read this intro on MDN.
In short:
There are multiple ways to encode the data sent in a POST request.
The way the receiver should deal with this data depends on how it's encoded by the sender.
The sender has to communicate the encoding with its request — usually via the Content-Type header field.
I won't go into the details of possible encodings — the referenced introductory material covers them, and you should do your own research on them, but to maybe recap what's written there, here is some perspective.
Back in the 80s and 90s the web was "static" and the dreaded era of JavaScript-heavy "web apps" did not yet come. "Static" means you could not run any code in the client's browser, and had to encode any communication with the server in terms of plain HTML.
An HTML document could have two ways to make the client rendering it to send something back to the server: a) embed an URL which would include query parameters; this would make the client to perform a GET request with these parameters sent to the server; b) embed an HTML "form" which, when "submitted", would result in performing a rather more complex POST request with the data taken from the filled in form.
The latter approach was the way to leverage the browser's ability to perform reasonably complex data processing — such as slurpling a file selected by the user in a specific form's control, encoding it appropriately and sending it to the server along with the other form's data.
There were two ways to encode the form's data, and they are both covered by the linked article, please read about them.
The crucial thing to understand about this "static web with forms" approach is that it worked like this: the server sends an HTML document containing a web form, the browser renders the document, the user fills the form in and clicks the "submit" button rendered by the browser; the browser collects the data from the form's controls, for entries of type "file" it reads and encodes the contents of those files and finally performs an HTTP POST request with this stuff encoded to the URL specified by the form. The server would typically respond with another HTML document and so on.
OK, so here came "web 2.0", and an "XHR" (XMLHttpRequest) was invented. It has "XML" in its name because that was the time when XML was perceived by some as a holy grail which would solve any computing problem (which it, of course, failed to do). That thing was invended to be able to send almost arbitrary data payloads; XML and JSON encoding were supported at least.
The crucial thing to understand is that this way to communicate with the server is completely parallel to the original one, and the only thing they share is that they both use HTTP POST requests.
By now you should possibly see the whole picture: contemporary JS libs allow you to contruct and perform any sort of request: they allow you to create a "web form"-style request or to create a JS object, and serialise it to JSON, and send the result in an HTTP POST request.
As you can see, any approach allows you to pass structured data containing multiple distinct pieces of data to the server, and the way to handle this all is a matter of agreement between the server and the client, that is, the API convention, if you want.
The difference between various approaches is that the web-form-style approach would take care of encoding the contents of the file for you, while if you opt to send your file in a JSON object, you'll need to encode it yourself — say, using base64 encoding.
Combined approaches are possible, too.
For instance, you can directly send binary data of a file as a POST request's body, and submit a set of parameters along with the request by encoding them as query-parameters of the URL. Again, it's up to the agreement between the client and the server about how the latter encodes the data to be sent and the former decodes them.
All-in-all, I'd recommend to take a pause and educate yourself on the stuff I have outlined above, and then have another stab at solving the problem, but this time — with reasonably complete understanding about how the stuff works under the hood, and how you intend to wield it.
This question already has answers here:
What exactly is an HTTP Entity?
(10 answers)
Closed 9 years ago.
Now I know what's an http entity. But what's entity used for?
I mean, when an application manipulates an http request or response, it just need to know how to parse message head and message body. Then what's the role of an entity? They have almost similar structures.
I dont really understand what you are trying to ask?
If you mean can we skip using HttpEntity in response and request at all? The answer is no!
its a convention you have to follow it, that how internet works!
Quoting entities from apache documentation:
Since an entity can represent both binary and character content, it
has support for character encodings (to support the latter, ie.
character content).
The entity is created when the request was successful, and used to
read the response.
To read the content from the entity, you can either retrieve the input
stream via the HttpEntity.getContent() method, which returns an
InputStream, or you can supply an output stream to the
HttpEntity.writeTo(OutputStream) method, which will return once all
content has been written to the given stream.
When the entity was received as a result of a response, the methods
getContentType() and getContentLength() methods are for reading the
common headers Content-Type and Content-Length respectively (if they
are available). Since the Content-Type header can contain a character
encoding for text mime-types like text/plain or text/html, the
getContentEncoding() method is used to read this information. If the
headers aren't available, a length of -1 will be returned, and NULL
for the content-type. If the Content-Type header is available, a
[Header] object will be returned.
When creating an entity for a request, this meta data has to be
supplied by the creator of the entity.
Other headers from the response are read using the getHeaders()
methods from the response object.
Source: http://wiki.apache.org/HttpComponents/HttpEntity
And I'm again sorry if I didn't get your question right, but hope this helps anyways.
What is the simplest way of sending HTTP POST requests and getting response (in XML format for example) using only Rebol3?
Is there an equivalent of using read/custom in Rebol2, as it is done in this question?
How to send an HTTP post with a custom header using REBOL
And where should I be donwnloading my Rebol3 binaries from? I've not found a lot of documentation on that...
The documentation at on Ports: Synchronous and Asynchronous Operations shows how to use both GET and POST. To summarize:
The default behavior is to assume the post data should be considered as application/x-www-form-urlencoded. (If you want to encode a block of ordinary Rebol data into that format, see %altwebform.r)
result: write http://www.rebol.com/cgi-bin/updata.r data
If you need a custom header, then instead of passing a string you need to pass a block. Start it with the WORD! post followed by a block of Rebol-formatted key/value pairs, and then your data:
result: write http://www.rebol.com/cgi-bin/updata.r compose [
post [
Content-type: "text/x-rebol"
;-- other fields here
]
(data)
]
The result will be in binary! and can be converted to string! to parse out any XML or whatever.
where should I be downloading my Rebol3 binaries from?
You should download binaries from http://www.rebolsource.net/
I'm trying to send a POST using Groovy HTTPBuilder but the data I want to send is already URL-encoded so I want HTTPBuilder to POST it as is. I tried the following:
def validationString = "cmd=_notify-validate&" + postData
def http = new HTTPBuilder(grailsApplication.config.grails.paypal.server)
http.request(Method.POST) {
uri.path = "/"
body = validationString
requestContentType = ContentType.TEXT
response.success = { response ->
println response.statusLine
}
}
But it gives me a NullPointerException:
java.lang.NullPointerException
at groovyx.net.http.HTTPBuilder$RequestConfigDelegate.setBody(HTTPBuilder.java:1200)
Since you're using pre-encoded form values you cannot use the default map-based content type encoder. You must specify the content type so the EncoderRegistry knows how to handle the body.
You may create the HttpBuilder with a content type that specifies the body is a URL-encoded string:
def http = new HTTPBuilder(url, ContentType.URLENC)
Or make the request passing the content type explicitly:
http.request(Method.POST, ContentType.URLENC) {
// etc.
For reference, here's how I figured it out--I didn't know before I read the question.
Looked at the request method's API docs to see what the closure was expected to contain. I've used HTTPBuilder only in passing, so I wanted to see what, specifically, the body "should" be, or "may" be, and if the two were different.
The four-arg version of the request method links to the RequestConfigDelegate class and said the options were discussed in its docs.
The RequestConfigDelegate.setBody method, which is what the body setter is, states the body "[...] may be of any type supported by the associated request encoder. That is, the value of body will be interpreted by the encoder associated with the current request content-type."
The request encoder link is to the EncoderRegistry class. It has an encode_form method taking a string and states it "assumes the String is an already-encoded POST string". Sounds good.
The request content-type link was to an HttpBuilder inner class method, RequestConfigDelegate.getRequestContentType, which in turn had a link to the ContentType enum.
That enum has a URLENC value that led me to believe it'd be the best first guess.
I tried the HTTPBuilder ctor taking a content type, and that worked.
Circled back to the request methods and noticed there's a version also taking a content type.
I'd guess the total time was ~5-10 minutes, much shorter than it took to type up what I did. Hopefully it'll convince you, though, that finding this kind of stuff out is possible via the docs, in relatively short order.
IMO this is a critical skill for developers to groom, and make you look like a hero. And it can be fun.
I am working on an app where we have to pass specific web api parameters to a web app using HTTP POST.
eg:
apimethod name
parameter1 value
parameter2 value
So do I use a string or URLEncodedPostData to send that data?
It would be good if u help me with a code eg.
I am using something like this but it doesnt post the data to the server.
Though the response code is ok/200 and I also get get a parsed html response when i read the httpresponse input stream. But the code doesnt post anything. So unable to get the expected response.
_postData.append("method", "session.getToken");
_postData.append( "developerKey", "value");
_postData.append( "clientID", "value");
_httpConnection = (HttpConnection) Connector.open(URL, Connector.READ_WRITE);
String encodedData = _postData.toString();
_httpConnection.setRequestMethod(HttpConnection.POST);
_httpConnection.setRequestProperty("User-Agent", "BlackBerry/3.2.1");
_httpConnection.setRequestProperty("Content-Language", "en-US");
_httpConnection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
_httpConnection.setRequestProperty("Content-Length",(new Integer(encodedData.length())).toString());
os = _httpConnection.openOutputStream();
os.write(requeststring.getBytes());`
The code you posted above looks correct - although you'll want to do a few more things (maybe you did this already but didn't include it in your code):
Close the outputstream once you've written all the bytes to it
Call getResponseCode() on the connection so that it actually sends the request
POSTed parameters are usually sent in the response BODY, which means URL-encoding them is inappropriate. Quote from the HTTP/1.1 protocol:
Note: The "multipart/form-data" type has been specifically defined
for carrying form data suitable for processing via the POST
request method, as described in RFC 1867 [15].
The post method allows you to use pretty arbitrary message bodies — so it is whatever format the server wants.