I'm trying to fetch utf-8 characters (accents) using node-soap but i'm getting strange chars (printed in console as '?'). I'm able to see accents in Soap UI.
soap.createClient('https://example.com/data.php?wsdl',options, (err, client) => {
if (err) return next(err);
client.getPerson({args}, (err, result) => {})
let name = result.data.name.$value;
//BUG name contains invalid chars instead of accents
});
� is the replacement character, set by js to figure any byte than can not be displayed : either a special ascii character (from \x00 to \x1F, and \x7F), or a non-ASCII byte (unless it is a part of an utf-8 character).
So, it seems that your SOAP response is not utf8-encoded.
Soap UI must be implemented so as to find out the original encoding and display it properly.
But your js engine does not, and replaces invalid utf8 characters in strings with �.
In node-soap, you can use in Client.service.port.method (and also in Client.method whereas it is not documented) an option from request module to get these characters encoded : for example
client.getPerson(
{args},
(err, result) => {},
{ encoding: 'latin1' } // you can add any option from request module ;
// `encoding` will point out the response encoding
)
Related
I try to post a form to server and here is the code:
ar request = new http.MultipartRequest("POST", _uri);
request.fields['user_acc'] = _userAcc;
// this issue should be solve
request.fields['user_nick_name'] = '中文名字';
request.fields['user_password'] = _password;
But the server side in the user_nick_name field always got null, note that is always, but I change it into English the server can receive that. I test on postman, the server can got Chinese correctly, so it's MultipartRequest issue on this problem.
My question is: Why the Dart or Flutter team so careless on this so important basic library? They even not consider about this simply issue. I opened a issue on github but no-one response, I think the team is done. So I ask the develop communit here, how to solve this problem anyway?
[UPDATE]
As kindly people suggested, I update my golang server now, if anyone else got this problem, you may wonna answer and suggestions too.
func HandleUserRegister(context *gin.Context) {
userAcc := context.PostForm("user_acc")
userAvatar := context.PostForm("user_avatar")
userNickName := context.PostForm("user_nick_name")
userPassword := context.PostForm("user_password")
userPhone := context.PostForm("user_phone")
userEmail := context.PostForm("user_email")
userGender := context.PostForm("user_gender")
userSign := context.PostForm("user_sign")
userType := context.PostForm("user_type")
userTypeInt, _ := strconv.Atoi(userType)
log.Infof("userAcc: %s, userNickName: %s, userPassword: %s", userAcc, userNickName, userPassword)}
This is based on gin, and this function is the api solver. If anyone wanna help, please help me figure it out.
OK! I update the question now, because it's really weird!. I did those test:
Post multiform via Flutter to Django server, it receives Chinese filed correctly;
Post multiform data via Postman, the golang(gin) server gots Chinese correctly;
Post multiform data via Flutter to golang(gin) server gots Chinese field null;
For more detail, I log the headers from my server for both postman(normal) and flutter (abnormal):
Postman:
request header: map[Content-Type:[multipart/form-data; boundary=--------------------------022341683711652813100488] Postman-Token:[855646d7-5bea-4b8f-b8df-81366226cd49] User-Agent:[PostmanRuntime/7.1.1] Content-Length:[422] Connection:[keep-alive] Cache-Control:[no-cache] Accept:[*/*] Accept-Encoding:[gzip, deflate]]
Flutter:
request header: map[User-Agent:[Dart/2.0 (dart:io)] Content-Type:[multipart/form-data; boundary=dart-http-boundary-.XUeYeqXpg4Yfyh8QhH1T5JB4zi_f3WxX9t7Taxhw91EFqhyki4] Accept-Encoding:[gzip] Content-Length:[574]]
Does anyone can notice the difference and let me know how to change the it make server can receive the Chinese Characters?
#DannyTuppeny is correct. This is a server problem.
When asked to include a non-ASCII field into a multi-part request, the Dart library correctly wraps this with a binary content-transfer-encoding.
String _headerForField(String name, String value) {
var header =
'content-disposition: form-data; name="${_browserEncode(name)}"';
if (!isPlainAscii(value)) {
header = '$header\r\n'
'content-type: text/plain; charset=utf-8\r\n'
'content-transfer-encoding: binary';
}
return '$header\r\n\r\n';
}
(Postman does not and simply sends the utf8 encoded string without any headers.)
Dart/ASCII looks like this:
--dart-http-boundary-HjDS88CmQicdgd8VaHSwPqJK8iR4H6rTG3LovSZy-QXGpU7pAB0
content-disposition: form-data; name="test"
stackover
--dart-http-boundary-HjDS88CmQicdgd8VaHSwPqJK8iR4H6rTG3LovSZy-QXGpU7pAB0
Dart/non-ASCII looks like this:
First boundary: --dart-http-boundary-58NU6u6_Fo22xjH8H7yPCtKuoKgB+A8+RTJ82iIK1gs3nnGMLlp\r\n
Encapsulated multipart part: (text/plain)
content-disposition: form-data; name="test"\r\n
content-type: text/plain; charset=utf-8\r\n
content-transfer-encoding: binary\r\n\r\n
Line-based text data: text/plain
\344\270\255\346\226\207\345\220\215\345\255\227
Boundary: \r\n--dart-http-boundary-58NU6u6_Fo22xjH8H7yPCtKuoKgB+A8+RTJ82iIK1gs3nnGMLlp\r\n
So the problem is that the server is unable to unwrap the value from the encapsulation.
EDIT
Here's the Postman trace I captured yesterday. It's multi-form, but fails to add the content-type-encoding header despite the field being non-ASCII.
MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "--------------------------595246000077585285134204"
[Type: multipart/form-data]
First boundary: ----------------------------595246000077585285134204\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="name"\r\n\r\n
Data (12 bytes)
0000 e4 b8 ad e6 96 87 e5 90 8d e5 ad 97 ............
Data: e4b8ade69687e5908de5ad97
[Length: 12]
Last boundary: \r\n----------------------------595246000077585285134204--\r\n
I tested by posting to httpbin and the response suggests that the characters were posted correctly:
"user_nick_name":"\u4e2d\u6587\u540d\u5b57"
I tried with both the Stable v1 SDK and a v2 SDK from Flutter. Is it possible the issue is on the server? Have you tried using something like Fiddler to capture what's actually being sent?
Edit: My guess is that your server side code is not correctly reading the data as MultipartForm data (eg. you should be using ParseMultipartForm and reading from MultipartForm).
The problem, it appears, is in formdata.go part of multipart. Go assumes that any multipart part with an Content-Type header is a file (not a field). However, knowing this you can change your server code as follows:
func main() {
r := gin.Default()
r.POST("/sotest", func(c *gin.Context) {
formValue := c.PostForm("form_value")
if formValue == "" {
formFile, _ := c.FormFile("form_value")
file, _ := formFile.Open()
b1 := make([]byte, formFile.Size)
file.Read(b1)
formValue = string(b1)
}
c.JSON(200, gin.H{
"status": "posted",
"formValue": formValue,
})
})
r.Run() // listen and serve on 0.0.0.0:8080
}
When you detect that PostForm returns the empty string, you know that Go has treated the field as a file, in which case you can Open and Read the 'file' and decode it as the utf-8 string that we know it is. Obviously, you could encapsulate the "try as PostForm and if that's empty, try as FormFile" test into a function.
If you don't want to have to test for empty string at the server, you could change your Dart end code to always utf-8 encode even non-ascii strings with
request.files.add(
new http.MultipartFile.fromBytes(
'some_form_value_name',
utf8.encode('the string value'),
contentType: new MediaType('text', 'plain', {'charset': 'utf-8'}),
),
);
and read them at the server with the Open/Read/string method.
I have now solved this. Thanks to Richard and Danny for their help.
1. Reason for this
No matter what happens but this really not only one-side problem, we can not say it's Flutter or Go wrong. But the combination, Flutter + Go server just may be got this issue. The behind reason I still not quit sure, but it must some head not right set (postman can do it right);
2. Solution
We don't only need know why but also how to solve it. Here is what I do:
Do not use the official http package. Using dio, which is a extension Dart package. link: https://pub.dartlang.org/packages/dio
It's more clean and easy to use, so my code becomes to:
FormData _formData = new FormData.from({
"user_acc": _userAcc,
"user_nick_name": _userNickName,
'user_password': _password,
});
Dio dio = new Dio();
Response response = await dio.post(usersUrl, data: _formData);
print(response.data);
I can not post the none-English words now:
INFO[0668] userAcc: ww, userNickName: 小鹿叮叮婴儿湿巾手口专用80抽湿纸巾婴儿湿巾婴儿100抽带盖批发【原价】34.90元【券后】9.9元【省】25元【复制此信息打开手机淘宝即可查看并下单】￥Tnsx0E77pFs￥【必买理由】新品预售80抽*3仙女联盟，更多优惠fd.loliloli.pro , userPassword: ww
INFO[0671] user exist.
I’m trying to upload an Sqlite database from IOS Swift 3 to my server using Alamofire 4.0, but having problems converting the sqlite file into the data type required to upload.
The majority of posts / question examples seem to default to uploading images, but I am struggling to find example of uploading sqlite or other file types (for back-up purposes)
I have searched for the basic code and found this so far which looks very reasonable (thanks to following post: Alamofire 4 upload with parameters)
let parameters = ["file_name": "swift_file.jpeg"]
Alamofire.upload(multipartFormData: { (multipartFormData) in
multipartFormData.append(UIImageJPEGRepresentation(self.photoImageView.image!, 1)!, withName: "photo_path", fileName: "swift_file.jpeg", mimeType: "image/jpeg")
for (key, value) in parameters {
multipartFormData.append(value.data(using: String.Encoding.utf8)!, withName: key)
}
}, to:"http://sample.com/upload_img.php")
{ (result) in
switch result
{
case .success(let upload, _, _):
upload.uploadProgress(closure: { (progress) in
//Print progress
})
upload.responseJSON { response in
//print response.result
}
case .failure(let encodingError):
//print encodingError.description
}
}
The part I’m struggling with is to append the sqlite file to the upload (multipartFormData.append(………..?) I’ve searched but not found any good reference posts.
Yes, i’m a newbe, but trying hard, any help would be appreciated…..
It's exactly the same as the image example except that the mime type would be application/octet-stream.
Also, you'd probably go ahead and load it directly from the fileURL rather than loading it into a Data first.
As an aside, the parameters in that example don't quite make sense, as it looks redundant with the filename provided in the upload of the image itself. So you'd use whatever parameters your web service requires, if any. If you have no additional parameters, you'd simply omit the for (key, value) { ... } loop entirely.
Finally, obviously replace the file field name with whatever field name your web service is looking for.
// any additional parameters that must be included in the request (if any)
let parameters = ["somekey": "somevalue"]
// database to be uploaded; I'm assuming it's in Documents, but perhaps you have it elsewhere, so build the URL appropriately for where the file is
let filename = "test.sqlite"
let fileURL = try! FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false)
.appendingPathComponent(filename)
// now initiate request
Alamofire.upload(multipartFormData: { multipartFormData in
multipartFormData.append(fileURL, withName: "file", fileName: filename, mimeType: "application/octet-stream")
for (key, value) in parameters {
multipartFormData.append(value.data(using: .utf8)!, withName: key)
}
}, to: urlString) { result in
switch result {
case .success(let upload, _, _):
upload
.authenticate(user: self.user, password: self.password) // only needed if you're doing server authentication
.uploadProgress { progress in
print(progress.fractionCompleted)
}
.responseJSON { response in
print("\(response.result.value)")
}
case .failure(let encodingError):
print(encodingError.localizedDescription)
}
}
Unrelated, but if you're ever unsure as to what mime type to use, you can use this routine, which will try to determine mime type from the file extension.
/// Determine mime type on the basis of extension of a file.
///
/// This requires MobileCoreServices framework.
///
/// - parameter url: The file `URL` of the local file for which we are going to determine the mime type.
///
/// - returns: Returns the mime type if successful. Returns application/octet-stream if unable to determine mime type.
func mimeType(for url: URL) -> String {
let pathExtension = url.pathExtension
if let uti = UTTypeCreatePreferredIdentifierForTag(kUTTagClassFilenameExtension, pathExtension as NSString, nil)?.takeRetainedValue() {
if let mimetype = UTTypeCopyPreferredTagWithClass(uti, kUTTagClassMIMEType)?.takeRetainedValue() {
return mimetype as String
}
}
return "application/octet-stream";
}
I've just set a Warning HTTP header for the first time. In the System.Net .NET namespace, there's a type WarningHeaderValue with this constructor:
WarningHeaderValue(int code, string agent, string text)
But it throws on my agent string saying
The format of value 'Company Name .NET Origin' is invalid.
What format is legal for the agent? I couldn't glean anything useful from the HTTP spec.
warn-agent = ( uri-host [ ":" port ] ) / pseudonym
; the name or pseudonym of the server adding
; the Warning header field, for use in debugging
; a single "-" is recommended when agent unknown
warn-text = quoted-string
warn-date = DQUOTE HTTP-date DQUOTE
What's missing in all these arcane specifications are examples.
In this case, quoted-string means include quotes within the string.
So instead of: "Deprecated API" , it would be "\"Deprecated API\""
Full example:
HttpResponseMessage response = await base.ExecuteAsync(cancellationToken);
string message = $"\"Deprecated API : use {NewUri} instead.\"";
response.Headers.Warning.Add(new WarningHeaderValue(299, "-", message));
return response;
I need to create a JSON object from a json string. I tried JSON.parse(json-string) but I get error:
SyntaxError: Unexpected token o
But when I run the same string in Android JSONObject(json-string) I get a no errors
HTTP.get(url, function (error, response) {
if (response) {
var content = JSON.parse(response);
console.log(content);
Am I doing something wrong? Doesn't response contains the json-string and nothing else un-modified? Thanks
response is an object. You need to JSON.parse(response.content) or just response.data.
From official docs:
Contents of the result object:
statusCode Number
Numeric HTTP result status code, or null on error.
content String
The body of the HTTP response as a string.
data Object or null
If the response headers indicate JSON content, this contains
the body of the document parsed as a JSON object.
headers Object
A dictionary of HTTP headers from the response.
I'm trying to figure out how the best way to easily send HTTP/HTTPS requests and to handle gzip/deflate compressed responses along with cookies.
The best I found was https://github.com/mikeal/request which handles everything except compression. Is there a module or method that will do everything I ask?
If not, can I combine request and zlib in some manner? I tried to combine zlib and http.ServerRequest, and it failed miserably.
For anyone coming across this in recent times, the request library supports gzip decompression out of the box now. Use as follows:
request(
{ method: 'GET'
, uri: 'http://www.google.com'
, gzip: true
}
, function (error, response, body) {
// body is the decompressed response body
console.log('server encoded the data as: ' + (response.headers['content-encoding'] || 'identity'))
console.log('the decoded data is: ' + body)
}
)
From the github readme https://github.com/request/request
gzip - If true, add an Accept-Encoding header to request compressed
content encodings from the server (if not already present) and decode
supported content encodings in the response. Note: Automatic decoding
of the response content is performed on the body data returned through
request (both through the request stream and passed to the callback
function) but is not performed on the response stream (available from
the response event) which is the unmodified http.IncomingMessage
object which may contain compressed data. See example below.
Note: as of 2019, request has gzip decompression built in. You can still decompress requests manually using the below method.
You can simply combine request and zlib with streams.
Here is an example assuming you have a server listening on port 8000 :
var request = require('request'), zlib = require('zlib');
var headers = {
'Accept-Encoding': 'gzip'
};
request({url:'http://localhost:8000/', 'headers': headers})
.pipe(zlib.createGunzip()) // unzip
.pipe(process.stdout); // do whatever you want with the stream
Here's a working example that gunzips the response
function gunzipJSON(response){
var gunzip = zlib.createGunzip();
var json = "";
gunzip.on('data', function(data){
json += data.toString();
});
gunzip.on('end', function(){
parseJSON(json);
});
response.pipe(gunzip);
}
Full code: https://gist.github.com/0xPr0xy/5002984
Check out the examples at http://nodejs.org/docs/v0.6.0/api/zlib.html#examples
zlib is now built into node.
Looking inside the source code - you must set the gzip param on the request lib itself for gzip to work. Not sure if this was intentional or not, but this is the current implementation. No extra headers are needed.
var request = require('request');
request.gzip = true;
request({url: 'https://...'}, // use encoding:null for buffer instead of UTF8
function(error, response, body) { ... }
);
All the answers here did not work and I was getting raw bytes back instead and the gzip flag was not working either. As it turns out you need to set the encoding to null to prevent requests from transforming the response to utf-8 encoding and instead keeps the binary response.
const request = require("request-promise-native");
const zlib = require("zlib");
const url = getURL("index.txt");
const dataByteBuffer = await request(url, { encoding: null });
const dataString = zlib.gunzipSync(response);