I'm trying to understand how HTTP file uploads work.
For instance, my VueJS app is calling a REST API (with Axios). When calling axios.request, no headers are set. There is just a FormData object containing the file to upload.
When the request arrives to the backend, I see that a Content-Type: multipart/form-data; ... header has been added to the request.
At which moment is this header created? Who is responsible for creating the header?
If it is a file upload Ajax request, in Axios, it's the browser that set the Content-Type: multipart/form-data;... header.
In Axios source code lib/adapters/xhr.js (the one that take charge of XMLHttpRequest), the HTTP request data will be checked. If it is an instance of FormData, then Content-Type header would be deleted and let browser do the job.
In lib/adapters/xhr.js (look at the comment in the source code):
if (utils.isFormData(requestData)) {
delete requestHeaders['Content-Type']; // Let the browser set it
}
For utils.isFormData(), the logic is:
// code in lib/utils.js
function isFormData(val) {
return (typeof FormData !== 'undefined') && (val instanceof FormData);
}
Related
I have a Node.js Express web server that returns an HTTP response JSON payload along with an error status (4xx or 5xx) when something goes wrong.
res.status(500).json({ error: 'message' });
From the Chrome browser developer console's Timing section, I can see a lot of time (up to 5 minutes) spent in the "Content Download" segment and ultimately I am getting "Failed to load response data" in the Response section after download fails.
Chrome developer console timing output
Other browsers like Firefox and Opera are able to successfully download the JSON payload successfully and display them in their respective developer consoles.
If I send back the HTTP status as 200, Chrome has no trouble downloading the payload.
Also, if I do not set the Cache-Control HTTP headers to "no-store, no cache...", Chrome is able to successfully download the payload with 4xx/5xx status. However, I would like to set this header as a good practice against cache misuse.
HTTP Response Headers in the success and failure case
Is there something specific I need to do for Chrome?
Thank you!
I just had a similar problem. For the request I used the fetch API and in case of an error I did not read the stream of the response body.
Neither the content of the response body was displayed in the devtools nor was the request included in the HAR export.
After debugging this in the console I noticed that the content is displayed in the response or preview tab as soon as I was reading the stream. (e.g.: await response.text())
Strangely enough the behavior changes as you described when the corresponding cache headers are not set.
In my case, it happened when nothing was awaiting the response body.
Before (no body is rendered in the preview tab):
async request(context: RequestOpts): Promise<Response> {
// ...
const response = await this.fetchApi(url, init);
if (response.status >= 200 && response.status < 300) {
return response;
}
throw new Error();
}
After:
async request(context: RequestOpts): Promise<Response> {
// ...
const response = await this.fetchApi(url, init);
if (response.status >= 200 && response.status < 300) {
return response;
}
throw await response.json();
}
Bear with me for any mistakes/wrong terminology since I am new to all this. I am using meteor to develop my project and i need to make a get request to an external API. (I already added meteor add http) Below is my code:
HTTP.call( 'GET', 'url', {}, function( error, response ) {
if ( error ) {
console.log( error );
} else {
console.log( response );
}
});
If i use the code inside my Client folder in Meteor I get the following error No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access meteor It has something to do with CORS which I didn't understand how to implement. If I use the code above in my Server side I do get the correct response in the console but how do I use it as a var on my client javascript code?
Tou can use .call function of HTTP and pass your header in options:
HTTP.call(method, url, [options], [asyncCallback])
Arguments
method String
The HTTP method to use, such as "GET", "POST", or "HEAD".
url String
The URL to retrieve.
asyncCallback Function
Optional callback. If passed, the method runs asynchronously, instead of synchronously, and calls asyncCallback. On the client, this callback is required.
Options
content String
String to use as the HTTP request body.
data Object
JSON-able object to stringify and use as the HTTP request body. Overwrites content.
query String
Query string to go in the URL. Overwrites any query string in url.
params Object
Dictionary of request parameters to be encoded and placed in the URL (for GETs) or request body (for POSTs). If content or data is specified, params will always be placed in the URL.
auth String
HTTP basic authentication string of the form "username:password"
headers Object
Dictionary of strings, headers to add to the HTTP request.
timeout Number
Maximum time in milliseconds to wait for the request before failing. There is no timeout by default.
followRedirects Boolean
If true, transparently follow HTTP redirects. Cannot be set to false on the client. Default true.
npmRequestOptions Object
On the server, HTTP.call is implemented by using the npm request module. Any options in this object will be passed directly to the request invocation.
beforeSend Function
On the client, this will be called before the request is sent to allow for more direct manipulation of the underlying XMLHttpRequest object, which will be passed as the first argument. If the callback returns false, the request will be not be send.
Souce: Here
Fixed it. On client side
Meteor.call("getURL",'url',{},function(err,res){
if(err){
console.log('Error: '+err);
}
if(!err){
console.log('Response: '+res);
}
and on server
Meteor.methods({
'getURL': function(url_l){
console.log("Request: "+url_l)
return HTTP.get(url_l)
}
});
I cannot interecept error from http post
a part of my mservice (http post method)
addApplicationLink(applicationLink: ApplicationLink){
let body = JSON.stringify(applicationLink);
let requestHeaders = new Headers();
var headers = new Headers();
headers.set('Content-Type', ['application/json; charset=utf-8']);
let reqoptions = new RequestOptions({
headers: headers
});
return this._http.post(this._applicationLinksUrl + this._linkServicePath,body,{headers: headers});
in my component :
addApplicationLink() {
//todo
this.addNewLink = false;
/* check if must be done after call real rest service */
//this.applicationLinks.push(this.applicationLinkAdd);
this._applicationLinkService.addApplicationLink(this.applicationLinkAdd)
.subscribe(data => {
this.applicationLinks.push(this.applicationLinkAdd)
},
error => {
// handle error
console.error('this an erreor ' + error.status)
}
)
When user tries to add two same applicationlinks , the backend returns an error 409
But when I execute , error.status displays 200 in browser console
I see also in browser console
XMLHttpRequest cannot load http://localhost:7001...... No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. The response had HTTP status code 409.
rem : Http post is made with json , thus there is a prefligth call
Have you an idea to intercept error 409 ?
In fact, your server doesn't send back the CORS header (Access-Control-Allow-Origin' header is missing). This prevent the browser from providing the actual 409 error to the Angular2 application within the browser.
You need to fix first the problem on the server and you will be able to see this 409 error.
For more details about how CORS works, you could have a look at this article:
http://restlet.com/blog/2015/12/15/understanding-and-using-cors/
I am trying to inject a new request header in the proxy request flow using JS policy to be sent to the backend server. When I look at the debug trace, I see that the json data in the request header is distorted.
I am trying to inject some string like
{"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}}
But when I look at the trace window I see this
{"scope":"","time_till":2264,id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM,"custom_data":{"c_id":"test_data"}}
what am I doing wrong?
var obj = {"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}};
var header_str = JSON.stringify(obj);
context.setVariable('json-header',header_str);
request.headers['x-json-hedar']= header_str;
I tested your code and it seems to work. Here's an example response where I set the header string as a response:
HTTP/1.1 200 OK
User-Agent: curl/7.30.0
Accept: */*
x-json-header: {"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}}
Content-Length: 0
It appears this is only an issue with the Apigee debug session / trace tool as the header value was set correctly. Here was the JSON download of the debug session showing this header value:
{
"name": "x-json-header",
"value": "{\"scope\":\"\",\"time_till\":2264,id_1\":\"hUXLXVqpA1J4vA9sayk2UttWNdM,\"custom_data\":{\"c_id\":\"test_data\"}}"
}
You can see that the value passed to the UI for displaying the debug info has the malformed json:
id_1\":\"hUXLXVqpA1J4vA9sayk2UttWNdM,
This does not appear to be a problem with the Apigee debug/trace UI. I see the malformed JSON trickle down to my backend service.
Here is the header I'm trying to send -
{"timeStamp":"2349218349381274","latitude":"34.589","longitude":"-37.343","clientIp":"127.0.0.0","deviceId":"MOBILE_TEST_DEVICE_AGAIN","macAddress":"23:45:345:345","deviceType":"phone","deviceOS":"iOS","deviceModel":"iPhone 5S","connection":"5G","carrier":"Vodafone","refererURL":"http://www.google.com","xforwardedFor":"129.0.0.0","sessionId":"kfkls498327ksdjf","application":"mobile-app","appVersion":"7.6.5","serviceVersion":"1.0","userAgent":"Gecko"}
But Apigee reads the header as below. Note the missing start quotes from some fields.
{"timeStamp":"2349218349381274",latitude":"34.589,longitude":"-37.343,clientIp":"127.0.0.0,deviceId":"MOBILE_TEST_DEVICE_AGAIN,macAddress":"23:45:345:345,deviceType":"phone,deviceOS":"iOS,deviceModel":"iPhone 5S,connection":"5G,carrier":"Vodafone,refererURL":"http://www.google.com,xforwardedFor":"129.0.0.0,sessionId":"kfkls498327ksdjf,application":"mobile-app,appVersion":"7.6.5,serviceVersion":"1.0,"userAgent":"Gecko"}
The header is used in a service callout to a backend service which parses it. And rightly so, I get the below error -
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('l' (code 108)): was expecting double-quote to start field name
at [Source: java.io.StringReader#22549cdc; line: 1, column: 35]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1378)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:520)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleUnusualFieldName(ReaderBasedJsonParser.java:1275)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._parseFieldName(ReaderBasedJsonParser.java:1170)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:611)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:301)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2796)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1942)
I encounter strange behaviour when adding JSON to a context variable for example like the following:
var header_str = JSON.stringify(obj);
context.setVariable('json-header',header_str);
I appreciate this is an example so you may not have included the full extent of the problem but this normally works (now it is not added to a variable first):
request.headers['x-json-header'] = JSON.stringify(obj);
Code like this also works if you can send the request from JavaScript
var headers = {"Accept": "application/json", "Accept-Language": "en"};
var sessionRequest = new Request(url, 'POST', headers, body);
var exchange = httpClient.send(sessionRequest);
exchange.waitForComplete()
if (exchange.isSuccess()){
var responseObj = exchange.getResponse().content.asJSON;
if (responseObj.error){
request.content += JSON.stringify(responseObj);
}
}
Also, I have had success with using an AssignMessage policy to build a request, followed by a Callout policy to read the stored request and then make that request and store the result in a response object which can then be read by an Extract Variables policy.
I'm trying to figure out how the best way to easily send HTTP/HTTPS requests and to handle gzip/deflate compressed responses along with cookies.
The best I found was https://github.com/mikeal/request which handles everything except compression. Is there a module or method that will do everything I ask?
If not, can I combine request and zlib in some manner? I tried to combine zlib and http.ServerRequest, and it failed miserably.
For anyone coming across this in recent times, the request library supports gzip decompression out of the box now. Use as follows:
request(
{ method: 'GET'
, uri: 'http://www.google.com'
, gzip: true
}
, function (error, response, body) {
// body is the decompressed response body
console.log('server encoded the data as: ' + (response.headers['content-encoding'] || 'identity'))
console.log('the decoded data is: ' + body)
}
)
From the github readme https://github.com/request/request
gzip - If true, add an Accept-Encoding header to request compressed
content encodings from the server (if not already present) and decode
supported content encodings in the response. Note: Automatic decoding
of the response content is performed on the body data returned through
request (both through the request stream and passed to the callback
function) but is not performed on the response stream (available from
the response event) which is the unmodified http.IncomingMessage
object which may contain compressed data. See example below.
Note: as of 2019, request has gzip decompression built in. You can still decompress requests manually using the below method.
You can simply combine request and zlib with streams.
Here is an example assuming you have a server listening on port 8000 :
var request = require('request'), zlib = require('zlib');
var headers = {
'Accept-Encoding': 'gzip'
};
request({url:'http://localhost:8000/', 'headers': headers})
.pipe(zlib.createGunzip()) // unzip
.pipe(process.stdout); // do whatever you want with the stream
Here's a working example that gunzips the response
function gunzipJSON(response){
var gunzip = zlib.createGunzip();
var json = "";
gunzip.on('data', function(data){
json += data.toString();
});
gunzip.on('end', function(){
parseJSON(json);
});
response.pipe(gunzip);
}
Full code: https://gist.github.com/0xPr0xy/5002984
Check out the examples at http://nodejs.org/docs/v0.6.0/api/zlib.html#examples
zlib is now built into node.
Looking inside the source code - you must set the gzip param on the request lib itself for gzip to work. Not sure if this was intentional or not, but this is the current implementation. No extra headers are needed.
var request = require('request');
request.gzip = true;
request({url: 'https://...'}, // use encoding:null for buffer instead of UTF8
function(error, response, body) { ... }
);
All the answers here did not work and I was getting raw bytes back instead and the gzip flag was not working either. As it turns out you need to set the encoding to null to prevent requests from transforming the response to utf-8 encoding and instead keeps the binary response.
const request = require("request-promise-native");
const zlib = require("zlib");
const url = getURL("index.txt");
const dataByteBuffer = await request(url, { encoding: null });
const dataString = zlib.gunzipSync(response);