I am decoding form fields submitted via a HTTP POST request using request-toolbelt. I successfully instantiated a MultipartDecoder like described here. Now I would like to access the form fields by the name I have given them when sending the request.
I am able to get the name of a field like this
from requests_toolbelt.multipart import decoder
multipart_string = b"--ce560532019a77d83195f9e9873e16a1\r\nContent-Disposition: form-data; name=\"author\"\r\n\r\nJohn Smith\r\n--ce560532019a77d83195f9e9873e16a1\r\nContent-Disposition: form-data; name=\"file\"; filename=\"example2.txt\"\r\nContent-Type: text/plain\r\nExpires: 0\r\n\r\nHello World\r\n--ce560532019a77d83195f9e9873e16a1--\r\n"
content_type = "multipart/form-data; boundary=ce560532019a77d83195f9e9873e16a1"
decoder = decoder.MultipartDecoder(multipart_string, content_type)
field_name = decoder.parts[0].headers[b'Content-Disposition'].decode().split(';')[1].split('=')[1]
But this seems quite wrong. What is the usual way to access the form field names?
I use it in the following to decode the result of that method:
lst = []
for part in decoder.MultipartDecoder(postdata.encode('utf-8'), content_type_header).parts:
disposition = part.headers[b'Content-Disposition']
params = {}
for dispPart in str(disposition).split(';'):
kv = dispPart.split('=', 2)
params[str(kv[0]).strip()] = str(kv[1]).strip('\"\'\t \r\n') if len(kv)>1 else str(kv[0]).strip()
type = part.headers[b'Content-Type'] if b'Content-Type' in part.headers else None
lst.append({'content': part.content, "type": type, "params": params})
I assume because its a standard Mime header, there are functions that can do the same, but with less code as well.
Related
Hopefully a quick question, I'm trying to connect to the KuCoin API, not super relevant as I think this is more an issue with how I'm using the POST function and how it sends JSON along
Here is my function that is supposed to place an order:
API.Order <- function(pair,buysell,price,size) {
path = "/api/v1/orders"
now = as.integer(Sys.time()) * 1000
json <- list(
clientOid = as.character(now),
side = buysell,
symbol=pair,
type="limit",
price=price,
size=size
)
json=toJSON(json, auto_unbox = TRUE)
str_to_sign = (paste0(as.character(now), 'POST', path, json))
signature = as.character(base64Encode(hmac(api_secret,str_to_sign,"sha256", raw=TRUE)))
passphrase=as.character(base64Encode(hmac(api_secret,api_passphrase,"sha256", raw=TRUE)))
response=content(POST(url=url,
path=path,
body=json,
encode="json",
config = add_headers("KC-API-SIGN"=signature,
"KC-API-TIMESTAMP"=as.character(now),
"KC-API-KEY"=api_key,
"KC-API-PASSPHRASE"=passphrase,
"KC-API-KEY-VERSION"="2")
),
"text",encoding = "UTF-8")
response
data.table(fromJSON(response)$data)
}
API.Order(pair,"sell",1.42,1.0)
And everything works, except I get the following response:
"{\"code\":\"415000\",\"msg\":\"Unsupported Media Type\"}"
Which is puzzling to me. Everything else checks out (the signature and other auth headers), and I set the encode to "json" in the POST.. I also can put it as standard "application/json" and neither works. I've been staring at this for hours now and I can't see what (likely very little) thing I got wrong?
Thanks
I'm trying to write a lambda that will return a .WAV file in chunks over HTTP. I've got my actual data in a byte slice (outputPayload [] byte) and am trying to pass it back. While the request seems to run, the response received is of a different length to what I expect and seems to be corrupted. Here's my code:
//Create the necessary headers
responseHeader := make(map[string]string)
responseHeader["Accept-Ranges"] = "bytes"
responseHeader["Content-Range"] = fmt.Sprintf("%s/%d", rangeRequired, fileSize)
responseHeader["Content-Type"] = fileType // this will be "audio/wav"
responseHeader["Content-Length"] = fmt.Sprintf("%d", returnedByteCount)
responseBody := string(outputPayload)
return events.APIGatewayProxyResponse{
StatusCode: http.StatusPartialContent,
Headers: responseHeader,
Body: responseBody,
}, nil
As a basic check, using more at the command line, the start of the original file looks like this:
RIFF$^?^C^#WAVEfmt ^P^#^#^#^A^#^B^#D<AC>^#^#^P<B1>^B^#^D^#^P^#data^#^?^C^#ESC^#^Y^#
While the downloaded file looks like this:
RIFF$^?^C^#WAVEfmt ^P^#^#^#^A^#^B^#D�^#^#^P�^B^#^D^#^P^#data^#^?^C^#ESC^#^Y^#
I'm guessing I have an encoding issue somewhere. My hunch is the string conversion is the problem, but that's the variable type I need for an APIGatewayProxyResponse "Body" component. How do I change my code output to ensure the payload matches the original file?
I am sending an email from a Shiny app via the SendGrid API using the HTTR package (using POST function with json encoding). I need to pass an R object in between the json quotes that are used to define the text body of the email:
I have tried to convert the R object to json as follows:
client_id<- "f432jj"
email_text<- paste("Below is your unique key:", client_id, "Please copy
your key to the clipboard, then click 'Begin'")
email_text<- jsonlite::toJSON(email_text)
Here is the json code into which I need to insert the email_text object.
body = '{"from": {"email":"xxx#gmail.com"},
"personalizations": [{"to": [{"email":"zzz#gmail.com"}],
"dynamic_template_data":{
"header":"A measure is ready to be completed",
"text": email_text,
"c2a_button":"Begin",
"c2a_link":"yyy#gmail.com"}}],
"template_id":"e-98766334"}'
When trying to pass in the email_text object as above and send the email, I get:
HTTP/1.1 400 Bad Request
I think this means the syntax is wrong.
Any help much appreciated.
Normally you wouldn't build your body JSON data as a string. You could build the list that represents the data and then let jsonlite turn it into a JSON string for you. Your example might look something like this
client_id<- "f432jj"
email_text<- paste("Below is your unique key:", client_id, "Please copy
your key to the clipboard, then click 'Begin'")
body <- list(
from = list(email="xxx#gmail.com"),
personalizations = list(list(to=list(list(email="zzz#gmail.com")))),
dynamic_template_data = list(
header="A measure is ready to be completed",
text = email_text,
c2a_button = "Begin",
c2a_link = "yyy#gmail.com"
),
template_id = "e-98766334"
)
jsonlite::toJSON(body,auto_unbox=TRUE)
I am using redis2-nginx-module to serve html content stored as a value in redis. Following is the nginx config code to get value for a key from redis.
redis2_query get $fullkey;
redis2_pass localhost:6379;
#default_type text/html;
When the url is hit the following unwanted response is rendered along with the value for that key.
$14
How to remove this unwanted output? Also if key passed as an argument doesn't exist in the redis, how to check this condition and display some default page?
(Here's a similar question on ServerFault)
There's no way with just redis2 module, as it always return a raw Redis response.
If you only need GET and SET commands you may try with HttpRedisModule (redis_pass). If you need something fancier, like hashes, you should probably try filtering the raw response from Redis with Lua, e.g. something along the lines of
content_by_lua '
local res = ngx.location.capture("/redis",
{ args = { key = ngx.var.fullkey } }
)
local body = res.body
local s, e = string.find(body, "\r\n", 1, true)
ngx.print(string.sub(body, e + 1))
';
(Sorry, the code's untested, don't have an OpenResty instance at hand.)
I have a configuration file in the format of blah = foo. I would like to have entries like:
http = https://stackoverflow.com/questions,header keys and values,string to search for.
I'm okay requiring that the the url be urlecncoded. Is there any ASCII character I can use that won't be valid value anywhere in the above example (After splitting once on =)? My example uses a comma but I think that is valid in a header value?
After pouring through some RFCs I figure someone is more familiar with this can save me some pain.
Also my project is in Go if there are existing std library that might help with this...
You can use a non-ascii character and urlencode, for example using the middle dot (compose + ^ + . on linux):
const sep = `·`
const t = `http = https://stackoverflow.com/questions·string to search for·header=value·header=value`
func parseLine(line string) (name, url, search string, headers []string) {
idx := strings.Index(line, " = ")
if idx == -1 {
return
}
name = line[:idx]
parts := strings.Split(line[idx+3:], sep)
if len(parts) < 3 {
// handle invalid line
}
url, search = parts[0], parts[1]
headers = parts[2:]
return
}
Although, using JSON is probably the best and most long-term maintainable option.
For completeness sake, a json version would look like:
type Site struct {
Url string
Query string
Headers map[string]string
}
const t = `[
{
"url": "https://stackoverflow.com/questions",
"query": "string to search for",
"headers": {"header": "value", "header2": "value"}
},
{
"url": "https://google.com",
"query": "string to search for",
"headers": {"header": "value", "header2": "value"}
}
]`
func main() {
var sites []Site
err := json.Unmarshal([]byte(t), &sites)
fmt.Printf("%+v (%v)\n", sites, err)
}
Essentially you have to look at RFC 3986, RFC 7230 and friends to see what can occur.
URIs are simple if you insist on them to be valid, just use the space character or "<" and ">" as delimiters.
Field values can be almost anything; HTTP forbids control characters though, so you might be able to use horizontal TABs (if you're ok with getting into trouble with invalid field values).