I am trying to send a byte array through POST using Node-RED. I can successfully create the buffer using this module and storing it in msg.payload. However I can't figure out how to add it as a parameter in a http request node.
The receiving application requires enclosing quotes. So I use the payload in the following url: localhost:port/path?var=\"{{payload}}\", but it gives
"Error converting http params to args: invalid character '\' looking for beginning of value"
If using it in the request url without quotes: localhost:port/path?var={{payload}} nothing gets through (I can see on the other end).
I am using Protobuf due to the application on the other side, but I've also tried creating a buffer, as described here. However, nothing changes.
POSTs should not have arguments in the URL. The data should all be in the body.
Do you need to make the msg.payload an object with keys matching the arg names.
msg.payload = {
var = [buffer]
}
You will probably have to play around with the content-type header as by default I believe Node-RED will send a JSON body and you probably want application/x-www-form-urlencoded
You can set the headers by adding a msg.headers object
Related
I'm trying to get live trading data from the Internet via HTTP, but it is updated continuously, so if I GET the data, it will keep downloading as long as there is data available. Until I stop the downloading stream, then I can access the data.
How to access the stream of data while the downloading is in progress?
I tried using Indy's TIdHTTP, so I can use SSL, but I tried the IdIOHandlerStream, but it was already used for IdSSLIOHandlerSocketOpenSSL. So I'm absolutely clueless here.
This is in response to a "multipart/form-data" request.
Please guide me...
Lrequest.Values['__RequestVerificationToken'] := RequestVerificationToken;
Lrequest.Values['acct'] := 'demo';
Lrequest.Values['pwd'] := 'demo';
try
Response.Text := Onhttp.Post('https://trading/data', Lrequest);
Form1.Memo1.Lines.Add(TimeToStr(Time) + ': ' + Response.Text);
except
on E: Exception do
Form1.Memo1.Lines.Add(TimeToStr(Time) + ': ' + E.ClassName +
' error raised, with message : ' + E.Message);
end;
UPDATE:
The data is an endless JSON string, like this:
{"id":"data","val":[{"rc":2,"tpc":"\\RealTime\\Global\\SGDIDR.FX","item":[{"val":{"F009":"10454.90","F011":"-33.1"}}]}]}
{"id":"data","val":[{"rc":2,"tpc":"\\RealTime\\Global\\SGDIDR.FX","item":[{"val":{"F009":"10458.80","F011":"-29.2"}}]}]}
and so on, and so on...
You can't use TIdIOHandlerStream to interface with a TCP connection, that is not what it is designed for. It is meant for performing I/O operations using user-provided TStream objects, ie for debugging previously captured sessions.
TIdHTTP is not really designed to handle endless HTTP responses in most cases, as you have described. What is the exact format that the server is delivering its live data as? What do the HTTP response headers look like? It is really difficult to answer your question without know the exact format being used.
However, that being said, there are some cases to consider, depending on what the server is actually sending:
if the server is using a MIME-based server-push format, like multipart/x-mixed-replace, you can enable the hoNoReadMultipartMIME flag in the TIdHTTP.HTTPOptions property, and then read the MIME data yourself from the TIdHTTP.IOHandler after TIdHTTP.Get() exits. For instance, you can use TIdMessageDecoderMIME to help you parse the MIME parts, see New TIdHTTP hoNoReadMultipartMIME flag in Indy's blog, or Delphi Indy TIdHttp and multipart/x-mixed-replace with Text and jpeg image.
Otherwise, if the server is using Transfer-Encoding: chunked, where each data update is sent as a new HTTP chunk, you can use the TIdHTTP.OnChunkReceived event. Or, you can enable the hoNoReadChunked flag in the TIdHTTP.HTTPOptions property, and then read the chunks yourself from the TIdHTTP.IOHandler after TIdHTTP.Get() exits. See New TIdHTTP flags and OnChunkReceived event in Indy's blog.
Otherwise, you could give TIdHTTP.Get() a TIdEventStream to write into, and then use that stream's OnWrite event to access the raw bytes. Or, you could write your own TStream-derived class that overrides the virtual Write() method. Either way, you would be responsible for manually parsing and buffering the raw body data as they are being written to the stream.
Otherwise, you may have to resort to using TIdTCPClient instead, implementing the HTTP protocol manually, then you would be solely responsible for reading in the HTTP response body however you want.
I have learned you can "decorate" the HTTP transport so that you can details of the request, however I can't quite figure out how you log the URL in the same line.
https://play.golang.org/p/g-ypQN9ceGa
results in
INFO[0000] Client request dns_start_ms=0 first_byte_ms=590 response_code=200 total_ms=590 url=
INFO[0000] 200
I'm perpetually confused if I should be using https://golang.org/pkg/context/#WithValue to pass around the context in a struct, especially in light where https://blog.golang.org/context-and-structs concludes with pass context.Context in as an argument.
Go through the behaviour of how the request is constructed in request.go from net/http. You see that the RequestURI field is never set there. Quoting from same reference,
Usually the URL field should be used instead.
It is an error to set this field in an HTTP client request
So, I would suggest you to use request.URL instead.
It is a parsed from the request uri. It should have the data you need.
You can construct the output as following:
f := log.Fields{
"url": fmt.Sprintf("%s %s%s", r.Method, r.URL.Host, r.URL.Path),
}
Also, in my experience, it is far more easier to use context.WithValue and pass the context as an argument.
Replace r.RequestURI by r.URL.String() in your code to log the full, valid URL (https://golang.org/pkg/net/url/#URL.String). RequestURI is empty on the client side (https://golang.org/pkg/net/http/#Request), as the output from your code is showing.
I don't see how context.Context relates to your question, but I believe https://blog.golang.org/context-and-structs is considered "best practice".
I'm using ASP.NET UrlRoutingModule directly (not through MVC) to map certain routes to their handlers:
RouteTable.Routes.Add(new Route("products/{name}", handler));
Then, at request time, I'm getting the values from each route:
RouteData routeData = HttpContext.Current.Request.RequestContext.RouteData;
routeData.Values.TryGetValue("name", out value);
Everything fine so far, I'm getting the proper values for each route. My problem is encoding: I want to get the raw value of a route data. Example: for the route above, if the requested URL is http://example.com/products/word%2Dword the resulted "name" is "word-word". What I want though is the exact value "word%2Dword".
I know that with ASP.NET I can get the raw unprocessed URL using Request.ServerVariables["HTTP_URL"] but unfortunately I cannot use this here.
Any help would be appreciated.
Thanks
EDIT
My specific problem is that I would like to get more products in a single request using their names. I have for example the following product names: "student,pupil" and "sick,ill" (their name contains a comma). I'm also using a comma to separate names in the request.
I handle the encoding on the client side so the GET request looks like this: http://example.com/products/student%2Cpupil,sick%2Cill (I'm encoding each name separately but I'm not encoding the separator).
On the server side the "name" parameter will be automatically decoded by ASP.NET and the result is: "student,pupil,sick,ill" so now I don't know which is the separator. Request.ServerVariables["HTTP_URL"] returns the URL as I want it ("products/student%2Cpupil,sick%2Cill") so I suppose there has to be a way to get the raw value as a route data.
The "raw value" you're seeing isn't actually the original value, it's what was encoded to make the URL safe for the HTTP Protocol.
http://example.com/products/word%2Dword for instance started out as http://example.com/products/word-word and is turned back into word-word as it comes out of the HTTP transport layer.
If you pass it through Server.URLEncode you will get back the same encoded value (%2D instead of -) - but if you can't use Server variables, are you going to have access to the Server object?
I am working on an app where we have to pass specific web api parameters to a web app using HTTP POST.
eg:
apimethod name
parameter1 value
parameter2 value
So do I use a string or URLEncodedPostData to send that data?
It would be good if u help me with a code eg.
I am using something like this but it doesnt post the data to the server.
Though the response code is ok/200 and I also get get a parsed html response when i read the httpresponse input stream. But the code doesnt post anything. So unable to get the expected response.
_postData.append("method", "session.getToken");
_postData.append( "developerKey", "value");
_postData.append( "clientID", "value");
_httpConnection = (HttpConnection) Connector.open(URL, Connector.READ_WRITE);
String encodedData = _postData.toString();
_httpConnection.setRequestMethod(HttpConnection.POST);
_httpConnection.setRequestProperty("User-Agent", "BlackBerry/3.2.1");
_httpConnection.setRequestProperty("Content-Language", "en-US");
_httpConnection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
_httpConnection.setRequestProperty("Content-Length",(new Integer(encodedData.length())).toString());
os = _httpConnection.openOutputStream();
os.write(requeststring.getBytes());`
The code you posted above looks correct - although you'll want to do a few more things (maybe you did this already but didn't include it in your code):
Close the outputstream once you've written all the bytes to it
Call getResponseCode() on the connection so that it actually sends the request
POSTed parameters are usually sent in the response BODY, which means URL-encoding them is inappropriate. Quote from the HTTP/1.1 protocol:
Note: The "multipart/form-data" type has been specifically defined
for carrying form data suitable for processing via the POST
request method, as described in RFC 1867 [15].
The post method allows you to use pretty arbitrary message bodies — so it is whatever format the server wants.
I am creating a HTTPS connection and setting the request property as GET:
_httpsConnection = (HttpsConnection) Connector.open(URL, Connector.READ_WRITE);
_httpsConnection.setRequestMethod(HttpsConnection.GET);
But how do I send the GET parameters?
Do I set the request property like this:
_httpsConnection.setRequestProperty("method", "session.getToken");
_httpsConnection.setRequestProperty("developerKey", "value");
_httpsConnection.setRequestProperty("clientID", "value");
or do I have to write to the output stream of the connection?
or do I need to send the Parameter/Values by appending it to the url?
Calling Connection.setRequestProperty() will set the request header, which probably isn't what you want to do in this case (if you ask me I think calling it setRequestHeader would have been a better choice). Some proxies may strip off or rewrite the name of non-standard headers, so you're better off sticking to the convention of passing data in the GET URL via URL parameters.
The best way to do this on a BlackBerry is to use the URLEncodedPostData class to properly encode your URL parameters:
URLEncodedPostData data = new URLEncodedPostData("UTF-8", false);
data.append("method", "session.getToken");
data.append("developerKey", "value");
data.append("clientID", "value");
url = url + "?" + data.toString();
HTTP GET send data parameters as key/value pairs encoded within URL, just like:
GET /example.html // without parameters
GET /example.html?Id= 1 // with one basic parameter
GET /example.html?Id=1&Name=John%20Doo // with two parameters, second encoded
Note follow rules for character separators:
? - split URL in two pieces: adddress to left and paremeters to right
& - must be used to separate on parameter from another
You must know your platform specific native string encode function. Javascript uses escape, C# uses HttpUtility.UrlEncode
Yep, headers and properties are pretty much all you can send in a GET. Also, you're limited to a certain number of characters, which is browser dependent - I seem to recall about 1024 or 2000, typically.