Good afternoon!
For many months I have not been able to generate a POST request to send the firmware.
My method:
Used fiddler to keep track of titles.
With disabled authorization, urllib3 sends the necessary files, but the fiddler could not see it
import urllib3
http = urllib3.PoolManager()
ip="10.254.255.105"
name_file="boot_v1.5_app_v2.27.8.fw"
uri = "/firmware/update_from_page.htm"
with open(name_file, 'rb') as fp:
file_data = fp.read()
send = http.request(
'POST',
'http://' + ip + uri,
fields={'new_firmware': (name_file, file_data),}
)
POSTMAN also worked even with enabled (digest) authorization. I found the correct header line ("Content-Type": "multipart/form-data; border=--------------------7e6a5e075c") needed to send a file with form function -data
Please help to form a request to send a file through the requests library. it is used in the rest of the code and supports the desired digital authorization.
Tried to use the requests library in different ways. I decided to reproduce the sending with disabled authorization based on the working code urllib3, but same errors.
"fields=" missing in the requests library replaced it with others and changed arguments with headers
send = requests.post(url='http://'+ip+uri,
# data={'new_firmware': file_data},
files={'new_firmware': file_data}
# verify=False,
)
date= {'new_firmware': file_data}or {'new_firmware': fp} and analogues - leads to freezing
files= {'new_firmware': (name_file, file_data),} and analogues = requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
I noticed the urllib3 header in Wireshark
POST /firmware/update_from_page.htm HTTP/1.1
Host: 10.254.255.105
Accept-Encoding: identity
Content-Length: 524476
Content-Type: multipart/form-data; boundary=4cf5492caddccc65a2b42d8bc691c30f
User-Agent: python-urllib3/1.26.14
Maybe it sends a file in jason format, but so far it has not been possible to figure it out, and so far I have not found working options
Related
I'm trying to download a firmware.bin file that is produced in a private Github repository. I have the code that is finding the right asset url to download the file and per Github instructions the accept header needs to be set to accept: application/octet-stream in order to get the binary file. I'm only getting JSON in response. If I run the same request through postman I'm getting a binary file as the body. I've tried downloading it using HTTPClient and I get the same JSON request. It seems the headers aren't being set as requested to tell Github to send the binary content as I'm just getting JSON. As for the ArduinoOTA abstraction, I can't see how to even try to set headers and in digging into the esp_https_ota functions and http_client functions there doesn't appear to be a way to set headers for any of these higher level abstractions because the http_config object has no place for headers as far as I can tell. I might file a feature request to allow for this, but am new to this programming area and want to check to see if I'm missing something first.
Code returns JSON, not binary. URL is github rest api url to the asset (works in postman)
HTTPClient http2;
http2.setAuthorization(githubname,githubpass);
http2.addHeader("Authorization","token MYTOKEN");
http2.addHeader("accept","application/octet-stream");
http2.begin( firmwareURL, GHAPI_CERT); //Specify the URL and certificate
With the ESP IDF HTTP client you can add headers to an initialized HTTP client using function esp_http_client_set_header().
esp_http_client_handle_t client = esp_http_client_init(&config);
esp_http_client_set_header(client, "HeaderKey", "HeaderValue");
err = esp_http_client_perform(client);
If using the HTTPS OTA API, you can register for a callback which gives you a handle to the underlying HTTP client. You can then do the exact same as in above example.
I'm trying to send AT commands to my ESP32* module and am not getting any response back.
I need to perform a POST request that contains the username and password and other requests later on. I am not structuring these correctly and there is not a lot of good documentation for this.
NOTE: because I cannot share my complete url due to privacy I will use something with the same length ********connected.com:443
Send login information to ********connected.com/login (POST) body{"email":"myemail.ca", "password":"xxxxx"}
once I get the token I will make other requests.
get information regarding user profile ********connected.com/getRoutine ( GET) query param username="bob"
I really want to understand how these requests are structured so if someone can explain it to me elegantly that would be great!
Here is what I have tried..
AT
OK
AT+CIPSTART="TCP","********connected.com",443
CONNECT
OK
AT+CIPSEND=48
> "GET ********connected.com:443/getUsersOnline"
OK
>
Recv 48 bytes
SEND OK
CLOSED
REQUESTED POST REQUEST I HAVE USED
AT+CIPSEND=177 “POST \r Host: ********connected.com\r\n Accept: application/json\r\n Content-Length: 224r\n Content-Type: application/jsonr\n { "email":"myemail.com", "password":"myPassword" } “
There are actually several parts of your system that might be the cause of the malfunctioning:
The AT commands sent (it is not clear how you check for server responses. Responses could proviede clues about what's wrong)
The server side app seems to be a custom implementation that might have bugs as well
The POST request might be malformed
Let's focus on the last one.
POST are described in RFC 7231, and though it is an obscure description without examples, it makes one thing clear: there's not actually a well defined standard... because it is strictly application dependant!
I also quote the relevant part of this brilliant answer to an old SO question:
When receiving a POST request, you should always expect a "payload", or, in HTTP terms: a message body. The message body in itself is pretty useless, as there is no standard.
For this reason, all we can do is to build a POST request as accurate as possible and then to debug the system as a whole thing making sure that the request matches what expected by the server side application.
In order to do this, let's check another external link I found: POST request examples. We found this request:
POST /test HTTP/1.1
Host: foo.example
Content-Type: application/x-www-form-urlencoded
Content-Length: 27
field1=value1&field2=value2
Now let's compare this example to your request:
POST
Host: ********connected.com
Accept: application/json
Content-Length: 224
Content-Type: application/jsonr
{ "email":"myemail.com", "password":"myPassword" }
You are saying to the server that you want to pass a resource to an unspecified application (no path), and that this resource is 224 bytes long (wrong! Message body is shorter).
For these reasons, at least these things can be improved:
POST /path/invic18app.php HTTP/1.1 //The path to the application and HTTP version are missing
Content-Length: 48 //This must be the length of the message body, without the header. I also would write it as the last option, just before message body
Write TWO empty lines before message body, otherwise the server will interpret it as further (wrong) options
I hope this helps you, even if it is a tentative answer (I cannot try this request myself). But, again, you definitely need to sniff packets a TCP levels, in order to avoid debugging the server if you are not sure that data is actually received! If you cannot install Wireshark, also tcpdump will be ok.
The code snippets generated from postman don't include whether it is HTTP or HTTPS
GET/apis/123?api-version=2017-03-01 HTTP/1.1
Host: contoso.anc.net
Content-Type: application/json
If-Match: *
Authorization: Bearer token
Cache-Control: no-cache
Postman-Token: fe465591-15b3-cbf0-9d9b-f2ed76efcb2c
The endpoint contoso.anc.net only responds to https endpoint and one of our customers had to figure out the hard way.
Is this missing information in the snippet or the service side should automatically bump to HTTPS when it gets a request over HTTP? Can somebody point to the spec?
Edit
This relates to the Generate Code Snippets that can be created within Postman using the code button.
The HTTP option:
By using console.log(pm.request) in the Tests tab you are able to see the request details in the Postman Console. This shows the Protocol as well as the other request data.
If the API is not telling you what protocol to use when making requests (In the documentation) or not giving the user any feedback within the response, that seems to me like a different problem your customer is facing.
Why .Net WebApi don't detect the request contentType automatically and do auto-binding?
If I make a request without informing the contentType a HTTP 500 error occour:
No MediaTypeFormatter is available to read an object of type 'ExampleObject' from content with media type ''undefined''.
why not try to detect the incoming data and bind automatically?
Another case:
This request with Content-Type: application/x-www-form-urlencoded send a JSON:
User-Agent: Fiddler
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Host: localhost:10329
Content-Length: 42
Request Body:
{"Name":"qq","Email":"ww","Message":"ee"}:
My Action don't detect the JSON request data automatically in object param:
public void Create(ExampleObject example) //example is null
{
{
Instead of letting the object null why they do not try to solve it?
Then, for the binding occurs I need to send with Content-Type: application/json.
It would be best if .Net WebAPI detects the type of request data and do a auto-binding? Why not in this way?
application/x-www-form-urlencoded means you will be sending data in the x-www-form-urlencoded standard. Sending data in another standard will not work.
Sounds like what you want to do is accept multiple formats from the server.
the way http works is that the client makes a request to the server for a resource and tells the server what content types it understands. This means that the client doesnt get a response it isnt able to decode, and the server knows which responses are more appropriate on the client. For example if you are a web-browser the most appropriate content type is text/html but if you get XML you can probably do something with that too. So you would make a request with the following:
accept: text/html, application/xml
this says you prefer html but also understand XML
In your example if your client wants application/x-www-form-urlencoded but can also deal with JSON then you should do the following when making a request
accept: application/x-www-form-urlencoded, application/json
For more details see the HTTP Spec on accept headers here http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
You may also want to create a new media type formatter so your server knows how to give clients application/x-www-form-urlencoded, take a look at this blog post for more info on how to do this http://www.strathweb.com/2012/04/rss-atom-mediatypeformatter-for-asp-net-webapi/
maybe you guys can help me with this. I am trying to implement
reCAPTCHA in my node.js application and no matter what I do, I keep
getting "invalid-site-private-key" as a response.
Here are the things I double and double checked and tried:
Correct Keys
Keys are not swapped
Keys are "global keys" as I am testing on localhost and thought it might be an issue with that
Tested in production environment on the server - same problem
The last thing I can think of is that my POST request to the reCAPTCHA
API itself is incorrect as the concrete format of the body is not
explicitly documented (the parameters are documented, I know). So this
is the request body I am currently sending (the key and IP is changed
but I checked them on my side):
privatekey=6LcHN8gSAABAAEt_gKsSwfuSfsam9ebhPJa8w_EV&remoteip=10.92.165.132& challenge=03AHJ_Vuu85MroKzagMlXq_trMemw4hKSP648MOf1JCua9W-5R968i2pPjE0jjDGX TYmWNjaqUXTGJOyMO3IKKOGtkeg_Xnn2UVAfoXHVQ-0VCHYPNwrj3PQgGj22EFv7RGSsuNfJCyn mwTO8TnwZZMRjHFrsglar2zQ&response=Coleshill areacce
Is there something wrong with this format? Do I have to send special
headers? Am I completely wrong? (I am working for 16 hours straight
now so this might be ..)
Thank you for your help!
As stated in the comments above, I was able to solve the problem myself with the help of broofa and the node-recaptcha module available at https://github.com/mirhampt/node-recaptcha.
But first, to complete the missing details from above:
I didn't use any module, my solution is completely self-written based on the documentation available at the reCAPTCHA website.
I didn't send any request headers as there was nothing stated in the documentation. Everything that is said concerning the request before they explain the necessary parameters is the following:
"After your page is successfully displaying reCAPTCHA, you need to configure your form to check whether the answers entered by the users are correct. This is achieved by doing a POST request to http://www.google.com/recaptcha/api/verify. Below are the relevant parameters."
-- "How to Check the User's Answer" at http://code.google.com/apis/recaptcha/docs/verify.html
So I built a querystring myself (which is a one-liner but there is a module for that as well as I learned now) containing all parameters and sent it to the reCAPTCHA API endpoint. All I received was the error code invalid-site-private-key, which actually (as we know by now) is a wrong way of really sending a 400 Bad Request. Maybe they should think about implementing this then people would not wonder what's wrong with their keys.
These are the header parameters which are obviously necessary (they imply you're sending a form):
Content-Length which has to be the length of the query string
Content-Type which has to be application/x-www-form-urlencoded
Another thing I learned from the node-recaptcha module is, that one should send the querystring utf8 encoded.
My solution now looks like this, you may use it or built up on it but error handling is not implemented yet. And it's written in CoffeeScript.
http = require 'http'
module.exports.check = (remoteip, challenge, response, callback) ->
privatekey = 'placeyourprivatekeyhere'
request_body = "privatekey=#{privatekey}&remoteip=#{remoteip}&challenge=#{challenge}&response=#{response}"
response_body = ''
options =
host: 'www.google.com'
port: 80
method: 'POST'
path: '/recaptcha/api/verify'
req = http.request options, (res) ->
res.setEncoding 'utf8'
res.on 'data', (chunk) ->
response_body += chunk
res.on 'end', () ->
callback response_body.substring(0,4) == 'true'
req.setHeader 'Content-Length', request_body.length
req.setHeader 'Content-Type', 'application/x-www-form-urlencoded'
req.write request_body, 'utf8'
req.end()
Thank you :)
+1 to #florian for the very helpful answer. For posterity, I thought I'd provide some information about how to verify what your captcha request looks like to help you make sure that the appropriate headers and parameters are being specified.
If you are on a Mac or a Linux machine or have access to one of these locally, you can use the netcat command to setup a quick server. I guess there are netcat windows ports but I have no experience with them.
nc -l 8100
This command creates a TCP socket listening on pot 8100 and will wait for a connection. You then can change the captcha verify URL from http://www.google.com/recaptcha/... in your server code to be http://localhost:8100/. When your code makes the POST to the verify URL you should see your request outputted to the scree by netcat:
POST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 277
Host: localhost:8100
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
privatekey=XXX&remoteip=127.0.0.1&challenge=03AHJYYY...&response=some+words
Using this, I was able to see that my private-key was corrupted.