When using httr::GET, in certain queries it replaces % with safe representation %25, but in other queries it doesn't. I cannot find any rule that would make this happen.
I'm using httr 1.4.1
Sample query where % is replaced (notice the error code and that URL entered is not the same as in response object returned):
> httr::GET("jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0")
Response [https://jira.spring.io/rest/api/latest/search?jql=project=Spring%2520Framework&startAt=0]
Date: 2020-01-16 22:57
Status: 400
Content-Type: application/json;charset=UTF-8
Size: 196 B
Query where it is not replaced (no error, URL in response same as entered):
> httr::GET("issues.jenkins-ci.org/rest/api/latest/search?jql=project='WEBSITE'%20OR%20project='Infrastructure'&startAt=0")
Response [https://issues.jenkins-ci.org/rest/api/latest/search?jql=project='WEBSITE'%20OR%20project='Infrastructure'&startAt=0]
Date: 2020-01-16 23:02
Status: 200
Content-Type: application/json;charset=UTF-8
Size: 430 kB
What is going on? Is it a bug in httr? Or should I change some parameters in GET() call?
tldr; use HTTPS requests with jira.spring.io to avoid a broken protocol upgrade.
It's not an R/HTTR issue. It's the website. Compare the results of HTTP ("failing with mystery %25") and HTTPS ("succeeding"):
http://jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0
{"errorMessages":["Error in the JQL Query: The character '%' is a reserved JQL character. You must enclose it in a string or use the escape '\u0025' instead. (line 1, character 15)"],"errors":{}}
https://jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0
{"errorMessages":["Error in the JQL Query: Expecting either 'OR' or 'AND' but got 'Framework'. (line 1, character 16)"],"errors":{}}
There appears to be a 'malfunction' in the HTTP -> HTTPS redirect protocol upgrade, which has this response header:
Status Code: 301 Moved Permanently
Location: https://jira.spring.io/rest/api/latest/search?jql=project=Spring%252520Framework&startAt=0
^^^^^
Thus a solution is to use the HTTPS endpoint and avoid the strange target Location..
Related
I'm doing a coding exercise developing a HTTP server using Go's net/http library. The server is supposed to pass a series of tests in a Gitlab pipeline. I have no access to these tests and I can't see how they are implemented.
The problem is that one test for an expected HTTP 204 No Content response fails as follows:
Expected an empty response body "", got "\n"
The way I build the response in my code is:
// w is the http.ResponseWriter of the handler function.
w.WriteHeader(http.StatusNoContent)
w.Header().Del("Content-Type")
w.Write(nil)
I also tried w.Write(make([]byte, 0)) with the same result.
I'm testing it locally with curl but I can't really see the characters that are being returned in the body:
$ curl -i --header "Content-Type: application/x-www-form-urlencoded" --request POST --data "PARAMETER=1" host:9000/path
HTTP/1.1 204 No Content
Date: Thu, 10 Sep 2020 16:12:21 GMT
$
Is the net/http server actually returning a carriage return, and how can I prevent this?. Thank you.
Sorry, I was looking at the wrong piece of code. Because I don't have any details about the tests, I don't really know what exact case is being tested. The comments above are correct, just using w.WriteHeader(http.StatusNoContent) doesn't produce any carriage return in the body. No need to delete Content-Type. My mistake was that I was using http.Error(w, "", http.StatusNoContent) instead.
What is minimal HTTP 200 OK Connection close response for Nginx/lua/openresty. I have:
local sock, err = ngx.req.socket(true)
sock:send("HTTP/1.1 200 OK\\r\\nConnection: close\\r\\n\\r\\n")
and curl says:
curl: (52) Empty reply from server
In a case of no response body, you should probably use 204 No Content response code; "201 Created" may be an option as well for requests that create resources.
Also: replace each double slash with a single one, as you don't need to escape slash to generate CR LF sequence.
I've just started working with the Quectel MC60 and I am having some issues:
About HTTP GET method, I make the following commands:
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QSSLCFG="https",1
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPGET=60
AT+QHTTPREAD=30
AT+QIDEACT
When using the QCOM software, I make a script running all the above commands sequentially. When it comes to the AT+QHTTPREAD command, the response is always "+CME ERROR: 3822" (HTTP response failed). What can it be? I'm sure the HTTP server is working properly.
The answer is that it is necessary to configure the request header
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPCFG="requestheader",1
AT+QHTTPPOST=77
GET path HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
AT+QHTTPREAD=30
AT+QIDEACT
NOTE: in AT+HTTPPOST=77, 77 is the size of the POST message (last two \r\n are required and count)
NOTE2: after GET you're supposed to write the path to the url inserted in AT+QHTTPURL. For example, if you specified your URL as https://www.my_host.com/debug/main/port, your AT+HTTPPOST request should look like this (don't forget the last two \r\n):
GET /debug/main/port HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
This is the code that provides the error, along with the output from it.
I'm positive my access keys and tokens are correct. I triple checked them.
I'm guessing my query may be wrong somehow? My guess was defaulting since_id=0 for my first run, but removal of that provides the same error.
mentions = GET(final_url, sig)
mentions
Response [https://api.twitter.com/1.1/search/tweets.json?q=#lolhauntzer&until=2016-01-20&since_id=0&result_type=recent&lang=en&count=100]
Date: 2016-01-19 05:09
Status: 401
Content-Type: application/json; charset=utf-8
Size: 64 B
Woops. Brain lapse. Need to replace the "#" in the URL with "%40". The "#" works on my other workstation though, which is kind of baffling right now.
Nginx is serving only static files, yet, some of file names contains '?'. Yes, the question mark.
All URLs that contains '?' yield 404 even though file actually exists. e.g.
> GET /foo?lang=ar.html HTTP/1.1
...
...
< HTTP/1.1 404 Not Found
While a file named foo?lang=ar.html does exists in the expected location.
> GET /foo%3flang=ar.html HTTP/1.1
...
...
< HTTP/1.1 200 OK
How do I write a rewrite directive so all '?' will be redirected to %3f?
You should url-encode your query string to escape special characters such as ? and =
Specifically, the name of your file you have to request for, once encoded, is this:
foo%3Flang%3Dar.html
In Javascript you can url-encode the filename with encodeURIComponent() function, in PHP you have urlencode().
You MUST encode the ? as %3F before the http call to nginx.
The reason is that the url rfc reserves the ? character as a special character (specifcally see section 3.3). Consequently nginx will, correctly, interpret an unescaped ? character as the end of the path part of the url