The question is simple. In AutoHotkey, how would you parse a raw HTTP request to access data like the method, http version, cookies and host.
;Example of a raw http request:
rawHttp =
(
POST https://www.example.com/Login HTTP/1.1
Host: www.example.com
Connection: Keep-Alive
Cookie: session=b5j2h46fdthr46t74g5g234g5f3g6753kj73l
username=lalala&password=12345&rememberMe=1
)
;Example usage of the function below:
httpObj := HttpTxtToObj(rawHttp)
MsgBox % "Request method: " httpObj.requestLine.method
MsgBox % "Request URI: " httpObj.requestLine.uri
MsgBox % "Request http version: " httpObj.requestLine.httpVersion
MsgBox % "Request header (Host): " httpObj.headers["Host"]
MsgBox % "Request header (Connection): " httpObj.headers["Connection"]
MsgBox % "Request header (Cookie): " httpObj.headers["Cookie"]
MsgBox % "Request body: " httpObj.body
;This function takes care of everything:
HttpTxtToObj(rawHttp) {
;Split request into "request line", "headers" and "body"(if existent)
RegExMatch(rawHttp, "OU)^(?P<requestLine>.+)\R(?P<headers>.+)\R\R(?P<body>.*)$",request)
If !request.Count()
RegExMatch(rawHttp, "OU)^(?P<requestLine>.+)\R(?P<headers>.+)$",request)
;Split request line into "method" "requestUrl" and "httpVersion"
RegExMatch(request.requestLine, "OU)^(?P<method>[^\s]+)\s(?P<uri>[^\s]+)\s(?P<httpVersion>.+)$", requestLine)
;Make a nice key value array for the headers:
headers := {}
While (p := RegexMatch(request.headers, "OSU)(?P<key>[^:]+):\s(?P<value>[^\R]+(\R|$))", currentHeader, p?p+1:1))
headers.Insert(currentHeader.key, currentHeader.value)
;The body is usually a query string, json string or just text. When you uplaod a file it may even contain binary data.
;All that would make the code quite a bit more complex, so for now we'll just pretend it's normal text, nothing special.
;Now lets return a nice multidimensional array
Return {requestLine: requestLine, headers: headers, body: request.body}
}
Related
I created a simple http2 server,
If I send a request to it with curl, it responds with some headers, although I did not set them explicity. How can I acces them inside the requesthandling function ( sayhello )? My code ( I've never used golang before)
server.go
package main
import (
"net/http"
"strings"
"fmt"
"github.com/gorilla/mux"
"golang.org/x/net/http2"
)
func sayHello(w http.ResponseWriter, r *http.Request) {
message := r.URL.Path
message = strings.TrimPrefix(message, "/")
message = "Hello " + message
w.Header().Set("myFirst", "golangQuestion")
w.Write([]byte(message))
for k, v := range w.Header() {
fmt.Println("[RESPONSE][Header]", k,":", v)
}
}
func main() {
router := mux.NewRouter()
router.PathPrefix("/").HandlerFunc(sayHello) // catch everything else rule
var srv = &http.Server{
Addr: "127.0.0.1:8081",
}
http2.ConfigureServer(srv, nil)
srv.Handler = router
sslCert := "./ssl.cert"
sslKey := "./ssl.key"
if err := srv.ListenAndServeTLS(sslCert, sslKey); err != nil {
panic(err)
}
}
Sending request:
curl --head --insecure https://127.0.0.1:8081
HTTP/1.1 200 OK
Myfirst: golangQuestion
Date: Tue, 18 Jun 2019 09:18:29 GMT
Content-Length: 6
Content-Type: text/plain; charset=utf-8
I can see that some headers are sent back, the one which I set explicitly is also recieved, but the output of
go run server.go
[RESPONSE][Header] Myfirst : [golangQuestion]
How can I acces the other headers, which were not explicitly set, but recieved by curl as well? I loopd through w.Headers, but it did not contain the implicitly set headers
for k, v := range w.Header() {
fmt.Println("[RESPONSE][Header]", k,":", v)
}
My expectation that the output of go run server.go shall be something like this:
[RESPONSE][Header] Myfirst : [golangQuestion]
[RESPONSE][Header] Date: [2019.02.12 ]
[RESPONSE][Header] Content-Length: [6]
Those headers are sent automatically when you call ResponseWriter.Write(). Quoting from its doc:
// Write writes the data to the connection as part of an HTTP reply.
//
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// DetectContentType. Additionally, if the total size of all written
// data is under a few KB and there are no Flush calls, the
// Content-Length header is added automatically.
//
// Depending on the HTTP protocol version and the client, calling
// Write or WriteHeader may prevent future reads on the
// Request.Body. For HTTP/1.x requests, handlers should read any
// needed request body data before writing the response. Once the
// headers have been flushed (due to either an explicit Flusher.Flush
// call or writing enough data to trigger a flush), the request body
// may be unavailable. For HTTP/2 requests, the Go HTTP server permits
// handlers to continue to read the request body while concurrently
// writing the response. However, such behavior may not be supported
// by all HTTP/2 clients. Handlers should read before writing if
// possible to maximize compatibility.
Write([]byte) (int, error)
ResponseWriter.Header() contains only the headers set explicitly. The Content-Type and Content-Length were sent by w.Write().
Note: if you want to suppress such automatic headers, you have to set their values to nil, e.g.:
w.Header()["Date"] = nil
Also note that if you set the values of such headers manually, those values will be sent without being changed.
I am a Golang api that accept multipart/form-data requests. For some clients, however, it fails to parse the form because it doesn't like the boundary being used by the client.
The header from the client is:
Content-Type:[multipart/form-data; boundary================1648430772==]
I've narrowed this down to the ParseMediaType function in the mime package.
If I call:
bad := "multipart/form-data; boundary=1650458473"
d, params, err := mime.ParseMediaType(v)
if err != nil {
fmt.Println("err", err)
}
fmt.Println(d, params)
I get the err: mime: invalid media parameter.
Note that if I do this call with
multipart/form-data; boundary=3fc88aad6d1341a4921fd5ac9efe607c
it succeeds no problem.
According to the https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html spec, it looks to me like these are all valid characters for a boundary.
Is this a bug in the Go mime library? Or is this really an invalid boundary?
The rfc you linked to contains BNF for the boundary and multipart body, it does not contain the BNF for the Content-Type Header Field. So while = in boundary is just fine it's not fine in the parameter value of the Content-Type header. At least not unquoted.
So to fix your first example change the Content-Type to this:
multipart/form-data; boundary="===============1648430772=="
https://play.golang.org/p/3Iuk_ACZaQ
Your second example multipart/form-data; boundary=1650458473 seems to work fine.
https://play.golang.org/p/xJWwBa_QiP
Finally found the answer. In the RFC 2045 doc (https://www.ietf.org/rfc/rfc2045.txt) it states that certain values cannot be used as parameter values in the Content-Type header.
The pertinent section:
tspecials := "(" / ")" / "<" / ">" / "#" /
"," / ";" / ":" / "\" / <">
"/" / "[" / "]" / "?" / "="
; Must be in quoted-string,
; to use within parameter values
So you can use an equal sign, but only if it's quoted, so Go fails on the parsing. The client in this case is sending a technically-incorrect value for the boundary param.
How do I send a simple HTTP POST/GET SOAP request to my Sonos loudspeaker in Lua?
I have tried simple HTTP POST and GET requests with success, but I do not know where to start with SOAP requests.
Note: I am a newbie at this. I have never worked with a NodeMCU before nor have I programmed in Lua. I have experience in other languages though.
I know how to do it in C#, Java and PHP.
This works in Postman:
HTTP Headers:
SOAPAction:urn:schemas-upnp-org:service:AVTransport:1#Pause
Content-Type:text/xml; charset="utf-8"
Host:192.168.0.10:1400
BODY:
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><u:Pause xmlns:u="urn:schemas-upnp-org:service:AVTransport:1"><InstanceID>0</InstanceID></u:Pause></s:Body></s:Envelope>
What I did is this and it does not work:
sendRequest("192.168.0.10")
function sendRequest(url)
print("Sending request to Sonos Playbar...")
sk = net.createConnection(net.TCP, 0)
sk:on("receive", function(sck, c) print(c) end )
sk:on("connection", function(sck, c)
print("\r\n\r\n\r\n")
-- HTTP 405: Method not allowed
-- sck:send("POST / HTTP/1.1\r\nHost: "..url..":1400\r\nConnection: keep-alive\r\nAccept: */*\r\n\r\n")
-- HTTP 500, UPnP 402: Invalid arguments
-- sck:send("POST /MediaRenderer/AVTransport/Control HTTP/1.1\r\nHost: "..url..":1400\r\nSOAPAction:urn:schemas-upnp-org:service:AVTransport:1#Pause\r\nConnection: keep-alive\r\n\r\nAccept: */*\r\n\r\n")
local content = nil;
content = "POST /MediaRenderer/AVTransport/Control\r\n"
content = content.."Host:192.168.0.10:1400\r\n"
content = content.."Content-Type:text/xml; charset=utf-8\r\n"
content = content.."SOAPAction:urn:schemas-upnp-org:service:AVTransport:1#Pause\r\n"
content = content.."\r\n"
-- SOAP Body
content = content.."<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\""
content = content.." s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">"
content = content.."<s:Body>"
content = content.."<u:Pause xmlns:u=\"urn:schemas-upnp-org:service:AVTransport:1\">"
content = content.."<InstanceID>0</InstanceID>"
content = content.."</u:Pause>"
content = content.."</s:Body>"
content = content.."</s:Envelope>"
-- SOAP Body End
print(content.."\r\n\r\n\r\n")
sck:send(content);
end)
sk:connect(1400, url)
end
I am getting this response of my Sonos player:
HTTP/1.1 500 Internal Server Error
CONTENT-LENGTH: 347
CONTENT-TYPE: text/xml; charset="utf-8"
EXT:
Server: Linux UPnP/1.0 Sonos/34.16-37101 (ZPS9)
Connection: close
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<s:Fault>
<faultcode>s:Client</faultcode>
<faultstring>UPnPError</faultstring>
<detail>
<UPnPError xmlns="urn:schemas-upnp-org:control-1-0">
<errorCode>401</errorCode>
</UPnPError>
</detail>
</s:Fault>
</s:Body>
</s:Envelope>
What am I doing wrong? I copied and paste the text, basically. Maybe it is the order of headers? Maybe I am declaring the headers wrong or something?
I don't have a Sonos device to play with. Thus, this ain't a confirmed answer.
The string in your content variable is not a valid HTTP request. Sonos doesn't understand it as the error code 401 means "invalid action".
You need the separate HTTP headers with \r\n. An extra \r\n needs to be placed right before the HTTP body. Therefore, I'd expect that your content should be:
"POST http://192.168.0.10:1400/MediaRenderer/AVTransport/Control\r\n
SOAPAction:urn:schemas-upnp-org:service:AVTransport:1#Pause\r\n
Content-Type:text/xml; charset=\"utf-8\"\r\n
Host:192.168.0.10:1400\r\n\r\n
<s:Envelope xmlns:s=\"http://schemas.xml......"
Finally! I have it working! Below is the code to get it working:
sendRequest("192.168.0.10")
function sendRequest(url)
print("Sending request to Sonos Playbar...")
local content = nil;
content = "";
-- SOAP Body
content = content.."<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\""
content = content.." s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">"
content = content.."<s:Body>"
content = content.."<u:Pause xmlns:u=\"urn:schemas-upnp-org:service:AVTransport:1\">"
content = content.."<InstanceID>0</InstanceID>"
content = content.."</u:Pause>"
content = content.."</s:Body>"
content = content.."</s:Envelope>"
-- SOAP Body End
http.post("http://"..url..":1400/MediaRenderer/AVTransport/Control",
'Content-Type: text/xml; charset="utf-8"\r\n'..
'Host:'..url..':1400\r\n'..
'SOAPAction:urn:schemas-upnp-org:service:AVTransport:1#Pause\r\n',
content, function(code, data)
if(code < 0) then
print("HTTP request failed with code "..code)
else
print(code, data)
end
end)
end
I'm using Indy with Lazarus
Here is my code:
IdHTTP1.Request.ContentType := 'text/plain' ;
IdHTTP1.Response.ContentType := 'text/plain' ;
IdHTTP1.Response.Charset := 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' ;
IdHTTP1.Request.CharSet:= 'ISO-8859-1,utf-8;q=0.7,*;q=0.3 ' ;
IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP1.Get('http://192.168.25.965:8541/rest/SearchCard('+MYCARD+')',Stream) ;
If I start MYCARD with a letter, the server is picking up the full string. However, if I start with a number, it stops at the first letter.
MYCARD:= '12366854'; //works
MYCARD:= 'A125ASD555'; //Works
MYCARD:= '123YH963'; // The server only sees 123
What am I doing wrong?
First off, the two Request properties you are setting are meaningless in a GET request, and you should not be setting any Response properties at all.
// get rid of these assignments
//IdHTTP1.Request.ContentType := 'text/plain' ;
//IdHTTP1.Response.ContentType := 'text/plain' ;
//IdHTTP1.Response.Charset := 'ISO-8859-1,utf-8;q=0.7,*;q=0.3' ;
//IdHTTP1.Request.CharSet:= 'ISO-8859-1,utf-8;q=0.7,*;q=0.3 ' ;
IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP1.Get('http://192.168.25.965:8541/rest/SearchCard('+MYCARD+')', Stream);
Second, using the current version of Indy, I cannot reproduce your issue. TIdHTTP.Get() sends the specified URL as-is, it makes no assumptions about the characters in it (you are responsible for URL encoding). In my testing, 123YH963 works just fine. Here is the actual HTP request being sent:
GET /rest/SearchCard(123YH963) HTTP/1.1
Host: 192.168.25.965:8541
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/3.0 (compatible; Indy Library)
As you can see, the full MYCARD text is in the requested resource, as expected. So any truncation must be happening on the server side, not in TIdHTTP itself.
Are you sure you are formatting the URL correctly to begin with? Are you sure it should actually be sent like this:
/rest/SearchCard(123YH963)
And not something more like these instead?
/rest/SearchCard%28123YH963%29
/rest/SearchCard/123YH963
/rest/SearchCard?param=123YH963
Intuit offers these instructions for uploading attachments (which become Attachable objects that can be associated with one or more transactions).
I believe I'm using python's requests module (via rauth's OAuth1Session module—see below for how I'm creating the session object) to generate these requests. Here's the code leading up to the request:
print request_type
print url
print headers
print request_body
r = session.request(request_type, url, header_auth,
self.company_id, headers = headers,
data = request_body, **req_kwargs)
result = r.json()
print json.dumps(result, indent=4)
and the output of these things:
POST
https://quickbooks.api.intuit.com/v3/company/0123456789/upload
{'Accept': 'application/json'}
Content-Disposition: form-data; name="Invoice 003"; filename="Invoice 003.pdf"
Content-Type: application/pdf
<#INCLUDE */MyDir/Invoice 003.pdf*#>
{
"Fault": {
"type": "SystemFault",
"Error": [
{
"Message": "An application error has occurred while processing your request",
"code": "10000",
"Detail": "System Failure Error: Cannot consume content type"
}
]
},
"time": "[timestamp]"
}
I have confirmed (by uploading an attachment through the QBO web UI and then querying the Attachable object through the API) that application/pdf is included in the list of acceptable file types.
At sigmavirus24's suggestion, I tried removing the Content-Type line from the headers, but I got the same result.
Here's how I'm creating the session object (which, again, is working fine for other QBO v3 API requests of every type you see in Intuit's API Explorer):
from rauth import OAuth1Session
def create_session(self):
if self.consumer_secret and self.consumer_key and self.access_token_secret and self.access_token:
session = OAuth1Session(self.consumer_key,
self.consumer_secret,
self.access_token,
self.access_token_secret,
)
self.session = session
else:
raise Exception("Need four creds for Quickbooks.create_session.")
return self.session
What might I be missing here?
EDIT: current area of exploration is here; I just formed the header you see (that has the "INCLUDE" string there) directly. Perhaps I should be using rauth to attach the file...
Without being able to see what code you're using with requests, I'm going to take a shot in the dark and tell you to remove setting your own Content-Type. You probably don't want that. It looks like you want multipart/form-data and requests will set that on its own if you stop fighting it.
It looks like you're missing the boundaries that QuickBooks is expecting (based on what you linked).
---------------------------acebdf13572468
Content-Disposition: form-data; name="file_content_01"; filename="IMG_0771.jpg"
Content-Type: image/jpeg
<#INCLUDE *Y:\Documents\IMG_0771.jpg*#>
---------------------------acebdf13572468--
The first and last line above seem to be what you're missing.